Imaging apparatus and control method thereof

ABSTRACT

An imaging apparatus according to the present invention includes: an image sensor configured to capture a moving image, and at least one memory and at least one processor which function as: a determining unit configured to determine a plurality of scenes of the moving image on a basis of a time variation of a parameter corresponding to a frame of the moving image; an acquiring unit configured to acquire a characteristic amount of each of the plurality of scenes; and a generating unit configured to generate information in which a characteristic amount acquired by the acquiring unit is associated with each of the plurality of scenes.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to an imaging apparatus and a controlmethod thereof.

Description of the Related Art

In recent years, techniques for determining a tone map using metadata(static metadata) indicating a maximum brightness of an entire movingimage and performing HDR (high dynamic range) display in accordance withthe maximum brightness of the entire moving image are widely used. Inaddition, techniques for sequentially (on a per-frame or per-scenebasis) changing a display method of a moving image using metadata(dynamic metadata) including a maximum brightness of each frame or eachscene of the moving image are becoming widespread. Specifically,techniques are emerging which, by sequentially changing a tone map (atone curve) using dynamic metadata, sequentially changes HDR (highdynamic range) display in accordance with an upper limit displaybrightness, a maximum brightness of a frame or a scene of a movingimage, or the like. WO 2015/017314 discloses a method of generatingmetadata (a part of dynamic metadata) of each scene of a moving imageand adding the metadata to moving image data in a post-production imagedata editing process. WO 2015/017314 also discloses a technique usedwhen dividing one moving image (moving image data) into a plurality ofmoving images (moving image data) in a post-production image dataediting process for generating and adding static metadata and dynamicmetadata with respect to each of a plurality of moving image data.

However, with the technique disclosed in WO 2015/017314, when there isno post-production (an image data editing process), an inability toobtain dynamic metadata makes it difficult to sequentially change thedisplay method (HDR display using tone mapping or the like) of a movingimage. In addition, since the technique disclosed in WO 2015/017314 is atechnique applied when dividing a moving image, even by using thetechnique disclosed in WO 2015/017314 when connecting a plurality ofmoving images, it is difficult to generate and add static metadatasuitable for moving image data after the connection. Furthermore, it isdifficult to generate and add, with respect to moving image dataexpected to be connected (moving image data prior to connection),suitable static metadata (static metadata expected to be connected).

SUMMARY OF THE INVENTION

The present invention provides a technique that enables dynamic metadataand the like (a characteristic amount of each scene) to be acquiredwithout involving post-production.

An imaging apparatus according to the present invention includes: animage sensor configured to capture a moving image, and at least onememory and at least one processor which function as: a determining unitconfigured to determine a plurality of scenes of the moving image on abasis of a time variation of a parameter corresponding to a frame of themoving image; an acquiring unit configured to acquire a characteristicamount of each of the plurality of scenes; and a generating unitconfigured to generate information in which a characteristic amountacquired by the acquiring unit is associated with each of the pluralityof scenes.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing a configuration example of an imagingapparatus according to a first embodiment;

FIG. 2 is a diagram showing an example of a time variation of a framemaximum brightness value according to the first embodiment;

FIGS. 3A to 3D are diagrams showing an example of a time variation ofvarious parameters during photography according to the first embodiment;

FIG. 4 is a flow chart showing an example of a photography processaccording to the first embodiment;

FIG. 5 is a diagram showing an example of a time variation of a framemaximum brightness value and an aperture value according to a secondembodiment;

FIG. 6 is a diagram showing an example of a time variation of a framemaximum brightness value and an aperture value according to the secondembodiment;

FIG. 7 is a block diagram showing a configuration example of an imagingapparatus according to a third embodiment;

FIGS. 8A to 8D are diagrams showing an example of frame images accordingto the third embodiment;

FIG. 9 is a block diagram showing a configuration example of an imageprocessing apparatus according to the first embodiment;

FIG. 10 is a flow chart showing an example of a connected moving imagedata generation process according to the first embodiment;

FIG. 11 is a schematic view showing an example of moving image data thatis a connection target according to the first embodiment;

FIG. 12 is a schematic view showing an example of connected moving imagedata according to the first embodiment;

FIG. 13 is a block diagram showing a configuration example of an imagingapparatus according to the second embodiment;

FIG. 14 is a flow chart showing an example of a photography processaccording to the second embodiment;

FIGS. 15A to 15G are schematic views showing a specific example of thephotography process according to the second embodiment;

FIG. 16 is a block diagram showing a configuration example of theimaging apparatus according to the first embodiment;

FIG. 17 is a flow chart showing an example of the photography processaccording to the first embodiment;

FIG. 18 is a schematic view showing an example of a time-lapsephotography mode according to the first embodiment;

FIG. 19 is a schematic view showing an example of dynamic metadataaccording to the first embodiment;

FIG. 20 is a schematic view showing an example of a slow photographymode according to the first embodiment;

FIG. 21 is a schematic view showing an example of dynamic metadataaccording to the first embodiment;

FIG. 22 is a schematic view showing an example of a tone curve accordingto the first embodiment;

FIG. 23 is a schematic view showing an example of a special effectpriority mode according to the second embodiment;

FIG. 24 is a schematic view showing an example of dynamic metadataaccording to the second embodiment;

FIG. 25 is a schematic view showing an example of a time priority modeaccording to the second embodiment; and

FIG. 26 is a schematic view showing an example of dynamic metadataaccording to the second embodiment.

DESCRIPTION OF THE EMBODIMENTS First Embodiment

Hereinafter, a first embodiment of the present invention will bedescribed. FIG. 1 is a block diagram showing a configuration example ofan imaging apparatus 1100 according to the present embodiment. Theimaging apparatus 1100 includes an imaging optical system 1101, animaging element 1102, an imaging control unit 1103, a characteristicamount acquiring unit 1104, a scene determining unit 1105, a metadatagenerating unit 1106, a metadata adding unit 1107, an output unit 1108,a storage unit 1109, an output IF 1110, a CPU 1111, a RAM 1112, a ROM1113, and an operating unit 1114.

The imaging optical system 1101 forms an optical image representing anobject on the imaging element 1102. For example, the imaging opticalsystem 1101 includes a lens group such as a zoom lens and a focusinglens, an aperture adjusting device, and a shutter device.

The imaging element 1102 captures an object image (a moving imagerepresenting the object). Specifically, the imaging element 1102performs a photoelectric conversion process in which a captured opticalimage (light incident from the object via the imaging optical system1101) is converted into an analog electrical signal. In addition, theimaging element 1102 further performs an AD conversion process ofconverting the analog electrical signal obtained by the photoelectricconversion process into a digital electrical signal (image data of oneframe of a moving image; frame image data). Subsequently, the imagingelement 1102 outputs the frame image data obtained by the AD conversionprocess to the characteristic amount acquiring unit 1104 and the scenedetermining unit 1105. By repeating these processes, the imaging element1102 sequentially outputs a plurality of frame image data respectivelycorresponding to a plurality of frames of a moving image.

The imaging control unit 1103 controls imaging conditions of the imagingapparatus 1100. In the present embodiment, the imaging control unit 1103controls exposure (exposure conditions) of the imaging apparatus 1100 inaccordance with a user operation with respect to the imaging apparatus1100, a state of the imaging apparatus 1100, and the like. For example,the imaging control unit 1103 controls exposure by controlling aperture,a charge accumulation time of the imaging element, or the like.Specifically, the imaging control unit 1103 controls exposure bycontrolling a state of the imaging optical system 1101, processing bythe imaging element 1102, or the like in accordance with a gain value, ashutter speed, an aperture value, or the like designated by a user.Furthermore, the imaging control unit 1103 controls focusing of theimaging apparatus 1100 in accordance with a user operation with respectto the imaging apparatus 1100, a state of the imaging apparatus 1100,and the like. For example, the imaging control unit 1103 controlsfocusing by controlling a drive amount, a drive direction, or the likeof the focusing lens. Specifically, when an AF (automatic focusing)photography mode is being set, the imaging control unit 1103 adjusts thefocusing lens to a predetermined position and calculates a shape (adistribution) of contrast of a frame image (an image of one frame of amoving image). Subsequently, the imaging control unit 1103 performs AFcontrol by adopting a position where the contrast is highest among aplurality of positions in the frame image as a position (a focusingposition) where a luminous flux is to be brought into focus in theimaging element 1102.

The characteristic amount acquiring unit 1104 acquires a characteristicamount (a frame characteristic amount) of frame image data output fromthe imaging element 1102. Specifically, the characteristic amountacquiring unit 1104 acquires, as a frame characteristic amount, acharacteristic amount for determining a plurality of scenes of a movingimage and a characteristic amount for acquiring a characteristic amount(a scene characteristic amount) of each scene. The frame characteristicamount can also be described a “parameter corresponding to a frame”. Inthe present embodiment, the characteristic amount acquiring unit 1104acquires a maximum brightness value (a frame maximum brightness value)of frame image data from the frame image data and outputs the framemaximum brightness value to the scene determining unit 1105. The framemaximum brightness value is used as both a characteristic amount fordetermining a plurality of scenes and a characteristic amount foracquiring a scene characteristic amount.

The scene determining unit 1105 performs a scene determination processof determining a plurality of scenes of a moving image. The scenedetermination process can also be described a “scene division process ofdividing an entire period of a moving image into a plurality ofperiods”. A frame characteristic amount often varies (significantly)during a scene change. In consideration thereof, in the presentembodiment, the scene determining unit 1105 determines a plurality ofscenes on the basis of a time variation of a frame characteristic amount(a frame maximum brightness value) output from the characteristic amountacquiring unit 1104. The time variation of a frame characteristic amountcan also be described a “variation in the frame characteristic amountbetween time-sequential frames”. The scene determining unit 1105 outputsa result of the scene determination process to the metadata generatingunit 1106. In addition, the scene determining unit 1105 outputs, to themetadata generating unit 1106, the frame characteristic amount (theframe maximum brightness value) output from the characteristic amountacquiring unit 1104, and outputs, to the metadata adding unit 1107, theframe image data output from the imaging element 1102. Alternatively,the scene determining unit 1105 may perform various types of imageprocessing with respect to the frame image data output from the imagingelement 1102 and output the frame image data after the image processingto the metadata adding unit 1107. As the image processing, for example,a correction process of reducing distortion or noise attributable to theimaging optical system 1101 and the imaging element 1102 may beperformed or white balance adjustment, a color conversion process, gammacorrection, or the like may be performed.

The metadata generating unit 1106 generates information (dynamicmetadata) to be added to moving image data (data of a moving image) andoutputs the dynamic metadata to the metadata adding unit 1107.Specifically, the metadata generating unit 1106 acquires (determines) ascene characteristic amount of each scene of a moving image on the basisof information (a result of the scene determination process and a framecharacteristic amount (a frame maximum brightness value) of each frameof the moving image) output from the scene determining unit 1105.Subsequently, the metadata generating unit 1106 generates informationthat associates the acquired scene characteristic amount to each sceneas dynamic metadata.

The metadata adding unit 1107 generates moving image data made up of aplurality of frame image data sequentially output from the scenedetermining unit 1105 (the imaging element 1102), and adds the dynamicmetadata output from the metadata generating unit 1106 to the movingimage data. Subsequently, the metadata adding unit 1107 outputs themoving image data after adding the dynamic metadata thereto to theoutput unit 1108. For example, moving image data is data in a fileformat such as MPEG-4 AVC or HEVC (High Efficiency Video Coding), andthe metadata adding unit 1107 performs an encoding process for obtainingmoving image data in the file format. In addition, the metadata addingunit 1107 adds the dynamic metadata as SEI (Supplemental EnhancementInformation) specified in MPEG-4 AVC or HEVC.

The output unit 1108 outputs, to the storage unit 1109, the moving imagedata output from the metadata adding unit 1107 (the moving image dataafter adding the dynamic metadata thereto). The output unit 1108 mayindividually output the moving image data and the dynamic metadatawithout having the metadata adding unit 1107 add the dynamic metadata tothe moving image data. In this case, the metadata adding unit 1107 andthe output unit 1108 may or may not output the moving image data and thedynamic metadata in association with each other.

The storage unit 1109 is a random-access recording medium such as a CF(compact flash) card, and stores the moving image data output from theoutput unit 1108 (the moving image data after adding the dynamicmetadata thereto). The storage unit 1109 is detachable from the imagingapparatus 1100 and can be mounted to an apparatus other than the imagingapparatus 1100 (such as a personal computer). Alternatively, the storageunit 1109 may be an internal recording medium that is not attachable toand detachable from the imaging apparatus 1100.

The output IF 1110 outputs the moving image data stored by the storageunit 1109 (the moving image data after adding the dynamic metadatathereto) to an external apparatus (not illustrated). For example, themoving image data is output in a stream format in accordance with acommunication protocol in compliance with the HDMI (registeredtrademark) (High-Definition Multimedia Interface) standard. It should benoted that methods of transmitting moving image data and dynamicmetadata are not particularly limited. For example, a parameterspecified in SMPTE (Society of Motion Picture & Television Engineers) ST2094 may be transmitted as dynamic metadata. Specifically, Scene-MaxCLL(Maximum Content Light Level) specified in HDR10+ may be transmitted asdynamic metadata.

The CPU 1111 is connected via an internal bus (not illustrated) to otherblocks of the imaging apparatus 1100. The CPU 1111 controls processingof the imaging apparatus 1100. The RAM 1112 is connected via theinternal bus (not illustrated) to other blocks of the imaging apparatus1100. The RAM 1112 is used as a work area of the CPU 1111 and atemporary storage area for temporarily storing various data. The ROM1113 is connected via the internal bus (not illustrated) to other blocksof the imaging apparatus 1100. Firmware related to processing by the CPU1111, information related to processing by the CPU 1111, and the likeare recorded in the ROM 1113 in advance.

The operating unit 1114 is connected to the CPU 1111 via the internalbus (not illustrated). The operating unit 1114 is constituted by variousoperating members as an input unit for receiving user operations. Theoperating unit 1114 includes a photography start button for startingphotography, a selector switch for switching between automatic controland manual control of focusing operations, and a focus ring forperforming a focus adjustment operation. In addition, the operating unit1114 includes a touch panel and a liquid crystal panel (both notillustrated) and causes displayed function icons to act as variousfunction buttons. The function buttons include a photography startbutton, a moving image photography mode selection button, a whitebalance setting button, and an ISO sensitivity setting button. Movingimage photography modes include a manual exposure photography mode, anautomatic exposure photography mode, an MF (manual focusing) photographymode, an AF (automatic focusing) photography mode, a time-lapsephotography mode, and a custom mode.

Next, an example of a photography process of the imaging apparatus 1100will be described. FIG. 2 shows an example of a time variation of aframe maximum brightness value acquired by the characteristic amountacquiring unit 1104 in a case where the scene determination process bythe scene determining unit 1105 is not performed. FIG. 2 shows anexample in a case of obtaining moving image data A in which the numberof a photography start frame (a frame obtained at the start ofphotography; a first frame of a moving image) is “0” and the number of aphotography end frame (a frame obtained at the end of photography; alast frame of the moving image) is “N”. In this case, an entire periodof a moving image represented by the moving image data A is handled as aperiod of one scene A0, and both a maximum brightness value of themoving image data A and a maximum brightness value of the scene A0equals a brightness value AL_(MAX). Hereinafter, an example of a casewhere the moving image data A shown in FIG. 2 is obtained will bedescribed as an example of a case where the scene determination processby the scene determining unit 1105 is performed.

FIGS. 3A to 3D show an example of a time variation of various parameters(a frame maximum brightness value, a period of a determined scene, ascene characteristic amount, and the like) during photography in orderto obtain the moving image data A. In the present embodiment, it isassumed that a maximum value of a frame maximum brightness acquired withrespect to one scene or, in other words, a maximum brightness value (ascene maximum brightness value) of the moving image data of the scene isacquired as a scene characteristic amount. In FIGS. 3A to 3D, “Fr_(NOW)”denotes a frame number (the number of a frame) of a frame imagepresently being photographed. “AL_(MAX_NOW)” denotes a frame maximumbrightness value of the frame Fr_(NOW) (a frame with the frame numberFr_(NOW)) or, in other words, a maximum brightness value of the frameimage presently being photographed. “An” in “AnL_(MAX)” denotes a scenenumber (the number of a scene), and “AnL_(MAX)” denotes a scene maximumbrightness value of the scene An (a scene with the scene number An). Ascene A1 refers to a scene that starts from a frame 0 (a photographystart frame). The frame number is counted by the scene determining unit1105. For example, the scene determining unit 1105 is equipped with acounter that is incremented every time frame image data is acquired anduses a value of the counter as the frame number. The scene number iscounted by the metadata generating unit 1106. Details will be providedlater.

FIG. 4 is a flow chart showing an example of a photography process bythe imaging apparatus 1100. The photography process shown in FIG. 4includes a scene determination process and a dynamic metadata additionprocess (a process of adding dynamic metadata to moving image data). Thephotography process shown in FIG. 4 is started when the CPU 1111 detectsthat a photography start operation by the user has been received by theoperating unit 1114 and realized by the CPU 1111 by controlling therespective blocks of the imaging apparatus 1100. For example, thephotography start operation is a user operation in which the photographystart button of the operating unit 1114 is depressed in anon-photographing state of the imaging apparatus 1100.

In S1401, the characteristic amount acquiring unit 1104 startsacquisition of a frame maximum brightness value AL_(MAX_NOW) of frameimage data output from the imaging element 1102. The characteristicamount acquiring unit 1104 outputs the frame maximum brightness valueAL_(MAX_NOW) to the scene determining unit 1105. In addition, when theframe maximum brightness value AL_(MAX_NOW) is higher than a scenemaximum brightness value AnL_(MAX) of the scene An including a framewith the frame number Fr_(NOW), the characteristic amount acquiring unit1104 records the frame maximum brightness value AL_(MAX_NOW) as thescene maximum brightness value AnL_(MAX) in the RAM 1112. This processcan be described a “process of updating the scene maximum brightnessvalue AnL_(MAX) recorded in the RAM 1112 with the frame maximumbrightness value AL_(MAX_NOW)”. FIG. 3A shows a state where the framemaximum brightness value AL_(MAX_NOW) which is higher than a scenemaximum brightness value A1L_(MAX) (the scene maximum brightness valueof a scene A1) is detected and the scene maximum brightness valueA1L_(MAX) is updated. When the frame Fr_(NOW) is frame 0, thecharacteristic amount acquiring unit 1104 records the frame maximumbrightness value AL_(MAX_NOW) as the scene maximum brightness valueA1L_(MAX) in the RAM 1112.

In S1402, the scene determining unit 1105 determines whether or not ascene change is to be performed in the frame Fr_(NOW). Thisdetermination can also be described a “determination of whether or not aperiod of a moving image is to be divided in the frame Fr_(NOW)”, a“determination of whether or not to determine (finalize) a scene”, andthe like. When it is determined that a scene change is not to beperformed (S1402: No), the process is advanced to S1406. When it isdetermined that a scene change is to be performed (S1402: Yes), theprocess is advanced to S1403.

In S1403, the scene determining unit 1105 determines (finalizes) aperiod up to a frame Fr_(NOW)−1 preceding the frame Fr_(NOW) as a periodof a scene.

In the present embodiment, the processes of S1402 and S1403 areperformed such that a scene change is not performed at a time positionwhere the frame maximum brightness value does not vary but a scenechange is performed at a time position where the frame maximumbrightness value varies. For example, the processes of S1402 and S1403are performed such that a scene change is not performed at a timeposition where the frame maximum brightness value varies in a variationamount that is smaller than a threshold but a scene change is performedat a time position where the frame maximum brightness value varies in avariation amount that is larger than the threshold. Specifically, inS1402, the scene determining unit 1105 compares the frame maximumbrightness value AL_(MAX_NOW) of the frame Fr_(NOW) with a frame maximumbrightness value AL_(MAX_NOW-1) of the frame Fr_(NOW)−1. Subsequently,the processes of S1402 and S1403 are performed such that a scene changeis not performed when a difference (an absolute difference) between theframe maximum brightness value AL_(MAX_NOW) and the frame maximumbrightness value AL_(MAX_NOW-1) is smaller than a threshold but a scenechange is performed when the difference is equal to or larger than thethreshold. It should be noted that the threshold is not particularlylimited. The threshold may be a fixed value set in advance or a valuethat can be changed by the user.

FIG. 3B shows a state where a scene has been determined (finalized) at atiming of a frame Fr_(NOW)=M+1. In the example shown in FIG. 3B, uponacquiring frame image data of a frame number M+1, the scene determiningunit 1105 determines (finalizes) a period from the frame 0 to a frame Mas a period of the scene A1. In addition, the scene determining unit1105 outputs a scene start frame number (the number of a first frame ofa scene) 0 of the scene A1 and a scene end frame number (the number of alast frame of a scene) M of the scene A1 to the metadata generating unit1106. Furthermore, the scene determining unit 1105 outputs a scenedetermination signal indicating that a scene has been determined to thecharacteristic amount acquiring unit 1104.

Upon acquiring the scene determination signal, the characteristic amountacquiring unit 1104 changes an area of the RAM 1112 in which the scenemaximum brightness value AnL_(MAX) is recorded. Accordingly, the scenemaximum brightness value of each scene is to be individually recorded inthe RAM 1112. In the example shown in FIG. 3B, in a state where thescene maximum brightness value A1L_(MAX) is recorded in a predeterminedarea of the RAM 1112, a recording area of a scene maximum brightnessvalue A2L_(MAX) is selected and the frame maximum brightness valueAL_(MAX_NOW) is recorded in the RAM 1112 as the scene maximum brightnessvalue A2L_(MAX). Subsequently, when the frame maximum brightness valueAL_(MAX_NOW) is higher than the scene maximum brightness valueA2L_(MAX), the scene maximum brightness value A2L_(MAX) recorded in theRAM 1112 is updated with the frame maximum brightness valueAL_(MAX_NOW).

In S1404, the metadata generating unit 1106 generates metadata (a partof dynamic metadata) of one scene from the scene start frame number andthe scene end frame number acquired from the scene determining unit 1105and the scene maximum brightness value acquired from the RAM 1112. Themetadata of one scene includes a scene start frame number, a scene endframe number, a scene number, a scene maximum brightness value, and thelike. For example, the scene determining unit 1105 is equipped with acounter that is incremented every time a scene start frame number and ascene end frame number are acquired and uses a value of the counter as ascene number. The counter may be incremented with an update of the sceneend frame number acting as a trigger.

In the example shown in FIG. 3B, the metadata generating unit 1106determines the scene number A1 and acquires the scene maximum brightnessvalue A1L_(MAX) in accordance with the acquisition of the scene startframe number 0 and the scene end frame number M. In addition, themetadata generating unit 1106 generates data in which the scene startframe number 0, the scene end frame number M, the scene number A1, andthe scene maximum brightness value A1L_(MAX) are associated with oneanother as the metadata of the scene A1. Subsequently, the metadatagenerating unit 1106 outputs the metadata of the scene A1 to themetadata adding unit 1107. Every time a scene start frame number and ascene end frame number are acquired, the metadata generating unit 1106changes an area of the RAM 1112 from which a scene maximum brightnessvalue is to be read. Accordingly, the metadata generating unit 1106 canread a scene maximum brightness value corresponding to a scene that isdetermined by a scene start frame number and a scene end frame numberfrom the RAM 1112.

In S1405, the metadata adding unit 1107 adds the metadata (the metadataof one scene) acquired from the metadata generating unit 1106 to movingimage data (moving image data of one scene) constituted by a pluralityof frame image data sequentially output from the scene determining unit1105. In the example shown in FIG. 3B, the metadata of the scene A1 isadded to the moving image data of the scene A1. The moving image dataafter adding the metadata thereto is recorded in the storage unit 1109.In this case, it is assumed that metadata is added as the SEI of HEVC.In addition, as the moving image data after adding the metadata thereto,an HEVC file generated by an encoding process of HEVC is recorded in thestorage unit 1109. Although details will be provided later, theprocesses of S1402 to S1406 may be repeated. In S1405 for second andsubsequent times, with respect to a recorded moving image (a movingimage recorded in S1405 in the past), a current moving image (a movingimage of one scene) is coupled as a continuation (update of the recordedmoving image).

In S1406, the CPU 1111 determines whether or not the operating unit 1114has received a photography end operation by the user. For example, thephotography end operation is a user operation in which the photographyend button of the operating unit 1114 is depressed in a photographingstate of the imaging apparatus 1100. When it is determined that aphotography end operation has not been performed (S1406: No), theprocess is returned to S1402 and the processes of S1402 to S1406 arerepeated. FIG. 3C shows a state after the scene A1 is determine in aframe M+1. In FIG. 3C, “scene A2” represents a scene subsequent to thescene A1. In a similar manner to during the photography of the scene A1(in the manner described above), the characteristic amount acquiringunit 1104 records the scene maximum brightness value A2L_(MAX) of thescene A2 in the RAM 1112. When it is determined that a photography endoperation has been performed (S1406: Yes), the process is advanced toS1407. When a photography end operation has been performed, the scenedetermining unit 1105 outputs the scene start frame number of a lastscene (the last scene of a moving image) and the scene end frame numberof the last scene (the number of a photography end frame; the number ofthe last frame of the moving image) to the metadata generating unit1106.

In S1407, the metadata generating unit 1106 generates metadata (a partof dynamic metadata) of the last scene from the scene start frame numberand the scene end frame number acquired from the scene determining unit1105 and the scene maximum brightness value acquired from the RAM 1112.

FIG. 3D shows a state where photography of an image of a photography endframe N has ended and acquisition of moving image data A has ended. Inthe example shown in FIG. 3D, the metadata generating unit 1106determines a scene number A2 and acquires the scene maximum brightnessvalue A2L_(MAX) in accordance with the acquisition of a scene startframe number M+1 and a scene end frame number N. In addition, themetadata generating unit 1106 generates data in which the scene startframe number M+1, the scene end frame number N, the scene number A2, andthe scene maximum brightness value A2L_(MAX) are associated with oneanother as the metadata of the scene A2. Subsequently, the metadatagenerating unit 1106 outputs the metadata of the scene A2 to themetadata adding unit 1107.

In S1408, the metadata adding unit 1107 adds the metadata (the metadataof the last scene) acquired from the metadata generating unit 1106 tomoving image data (moving image data of the last scene) constituted by aplurality of frame image data sequentially output from the scenedetermining unit 1105. In the example shown in FIG. 3D, the metadata ofthe scene A2 is added to the moving image data of the scene A2. Themoving image data after adding the metadata thereto is recorded in thestorage unit 1109. When the process of S1405 has been performed, withrespect to a recorded moving image (a moving image recorded in S1405), acurrent moving image (a moving image of the last scene) is coupled as acontinuation (update of the recorded moving image).

As described above, according to the present embodiment, in the imagingapparatus, a plurality of scenes are determined on the basis of a timevariation of a frame maximum brightness value and a scene maximumbrightness value of each of the plurality of scenes is acquired. Inaddition, information (dynamic metadata) which associates the scenemaximum brightness value to each of the plurality of scenes isgenerated. In other words, dynamic metadata and the like can be acquiredwithout involving post-production (an image data editing process). As aresult, a display method of a moving image can be sequentially changedon the basis of dynamic metadata. For example, in HDR (high dynamicrange) display using tone mapping, a tone map can be sequentiallychanged on the basis of dynamic metadata.

While an example of repeating a process of recording moving image dataof a scene in the storage unit 1109 has been described with reference toFIG. 4, this example is not restrictive. For example, the metadataadding unit 1107 may add dynamic metadata to moving image datarepresenting an entire moving image so that metadata is added to aportion of each scene upon completion of photography of the movingimage, and record the moving image data after adding the dynamicmetadata thereto in the storage unit 1109. In this case, the metadataadding unit 1107 temporarily records the acquired moving image data (aplurality of frame image data) and the metadata in the RAM 1112.

While an example of identifying a scene by a frame number has beendescribed with reference to FIG. 4, this example is not restrictive. Forexample, a scene may be identified by a photography time or aphotography time point. Specifically, a scene photography start time maybe used in place of a scene start frame number and a scene photographyend time may be used in place of a scene start end number. The scenephotography start time is a time from a predetermined timing (a timingwhere photography of a moving image is started or the like) to a startof photography of a scene, and the scene photography end time is a timefrom the predetermined timing to an end of photography of the scene.

While an example of detecting a scene change from a variation in a framemaximum brightness value between consecutive frames has been describedwith reference to FIG. 4, a determination method of a scene is notlimited thereto and a parameter for determining a scene is not limitedto a frame maximum brightness value. A parameter for determining a scenemay differ from a frame characteristic amount for acquiring a scenecharacteristic amount. For example, a parameter for determining a scenemay be an average value of a frame maximum brightness value of acorresponding frame that is a frame corresponding to the parameter and aframe maximum brightness value of one or more frames that aretime-sequential with respect to the corresponding frame. Specifically,the scene determining unit 1105 may record the frame maximum brightnessvalue of the frame Fr_(NOW) and the frame maximum brightness values ofone or more time-sequential frames preceding the frame Fr_(NOW) in theRAM 1112. In addition, the scene determining unit 1105 may determine aplurality of scenes so that a scene change is performed when the averagevalue of the frame maximum brightness values varies in a variationamount that is larger than a threshold. Alternatively, the frame maximumbrightness values of time-sequential frames subsequent to the frameFr_(NOW) may be used. One of the frame maximum brightness values oftime-sequential frames preceding the frame Fr_(NOW) and the framemaximum brightness values of time-sequential frames subsequent to theframe Fr_(NOW) may be used, or the frame maximum brightness values oftime-sequential frames both preceding and subsequent to the frameFr_(NOW) may be used. There may be a plurality of parameters (parametersof a plurality of types) for determining a scene.

A frame characteristic amount is not limited to a frame maximumbrightness value and a scene characteristic amount is not limited to ascene maximum brightness value. For example, the frame characteristicamount may be another representative value (an average value, a minimumvalue, a mode, an intermediate value, or the like) or a histogram of abrightness value of frame image data. The scene characteristic amountmay be another representative value (an average value, a minimum value,a mode, an intermediate value, or the like) of a frame characteristicamount acquired with respect to a scene. A representative value (amaximum value, an average value, a minimum value, a mode, anintermediate value, or the like) of a brightness value of scene movingimage data (moving image data of a scene) may be acquired as scenecharacteristic amount from the scene moving image data instead ofdetermining the scene characteristic amount from a frame characteristicamount. A frame characteristic amount of one frame or a scenecharacteristic amount of one scene may include a plurality of values(values of a plurality of types).

Second Embodiment

A second embodiment of the present invention will be described below.Hereinafter, points (configurations, processes, and the like) thatdiffer from those of the first embodiment will be described in detailand descriptions of points that are the same as those of the firstembodiment will be omitted. In the first embodiment, an example of usinga frame maximum brightness value as a parameter for determining a scenehas been described. An imaging parameter often varies during a scenechange. In consideration thereof, in the present embodiment, an exampleof using an imaging parameter that is used when capturing a frame imageas a parameter for determining a scene will be described. Specifically,an example of using an aperture value as a parameter for determining ascene in a manual exposure photography mode will be described.

The imaging apparatus according to the present embodiment has a similarconfiguration to the imaging apparatus 1100 shown in FIG. 1 (the firstembodiment). However, the present embodiment differs from the firstembodiment in the scene determination process by the scene determiningunit 1105. Furthermore, in the present embodiment, the characteristicamount acquiring unit 1104 does not output a frame maximum brightnessvalue to the scene determining unit 1105. Instead, the imaging controlunit 1103 outputs an aperture value to the scene determining unit 1105.The imaging control unit 1103 may also output a gain value, a shutterspeed, and the like to the scene determining unit 1105.

An example of a processing flow related to the scene determinationprocess will be described. First, the operating unit 1114 receives amode change operation (a user operation) for changing to the manualexposure photography mode. In accordance with the mode change operation,the CPU 1111 controls the respective blocks of the imaging apparatus1100 and sets the manual exposure photography mode. Next, the operatingunit 1114 receives a photography start operation. In accordance with thephotography start operation, the CPU 1111 controls the respective blocksof the imaging apparatus 1100 and starts photography. Next, theoperating unit 1114 receives an aperture variation operation (a useroperation) for varying the aperture value. The imaging control unit 1103changes a state of the imaging optical system 1101 in accordance withthe aperture value after the variation and outputs the aperture valueafter the variation to the scene determining unit 1105. Next, the scenedetermining unit 1105 determines whether or not to determine (finalize)a scene in accordance with the acquired variation to the aperture value.In other words, the scene determining unit 1105 determines whether ornot a scene change is to be performed at the present timing inaccordance with the acquired variation to the aperture value.

FIG. 5 shows an example of a time variation of a frame maximumbrightness value and an aperture value in a case where the scenedetermination process by the scene determining unit 1105 is notperformed. FIG. 5 shows an example in a case of obtaining moving imagedata B in which the number of a photography start frame is “0” and thenumber of a photography end frame is “N”. In FIG. 5, upon photography ofan image of a frame M+1, the aperture value is varied from F2.2 to F2.0.However, since the scene determination process is not performed, anentire period of a moving image represented by the moving image data Bis handled as a period of one scene B0, and both a maximum brightnessvalue of the moving image data B and a maximum brightness value of thescene B0 equal a brightness value BL_(MAX). Hereinafter, an example of acase where the aperture value is varied in a similar manner to FIG. 5and the moving image data B shown in FIG. 5 is obtained will bedescribed as an example of a case where the scene determination processby the scene determining unit 1105 is performed.

FIG. 6 shows an example of a time variation of a frame maximumbrightness value and an aperture value in a case where the scenedetermination process by the scene determining unit 1105 is performed.In the present embodiment, the scene determining unit 1105 determines aplurality of scenes so that a scene change is performed at a timeposition where the aperture value varies. Therefore, as shown in FIG. 6,a period from the frame 0 to the frame M is determined as a period of ascene B1 and a period from the frame M+1 to the frame N is determined asa period of a scene B2. In addition, by a process similar to thataccording to the first embodiment, a brightness value B1L_(MAX) isacquired as a scene maximum brightness value of the scene B1, abrightness value B2L_(MAX) is acquired as a scene maximum brightnessvalue of the scene B2, and dynamic metadata including the scene maximumbrightness values B1L_(MAX) and B2L_(MAX) is generated.

As described above, according to the present embodiment, in the imagingapparatus, a plurality of scenes are determined on the basis of a timevariation of an imaging parameter (an aperture value) and dynamicmetadata is generated in a similar manner to the first embodiment. Inother words, dynamic metadata and the like can be acquired withoutinvolving post-production. As a result, a display method of a movingimage can be sequentially changed on the basis of dynamic metadata.

While an example where a manual exposure photography mode is set hasbeen described, an automatic exposure photography mode in which anaperture value is automatically varied may be set instead. In theautomatic exposure photography mode, the imaging control unit 1103refers to frame image data acquired from the imaging element 1102 andautomatically varies an aperture value (independent of a useroperation). Even when the automatic exposure photography mode is set, aplurality of scenes can be determined on the basis of a time variationof the aperture value in a similar manner to when the manual exposurephotography mode is set.

While an example in which a scene change is performed at a time positionwhere the aperture value varies (even by a slightest amount) has beendescribed, a scene change may be performed at a time position where theaperture value varies by a variation amount that is larger than athreshold. For example, control may be performed such that a scenechange is not performed when the aperture value varies by less than onestep (for example by ⅓ step) but a scene change is performed when theaperture value varies by one step or more.

An imaging parameter for determining a scene is not limited to anaperture value. For example, a plurality of scenes may be determined onthe basis of a time variation of ISO sensitivity, a shutter speed, afocus position, a focal length, white balance, an exposure value, or thelike. An exposure value can be calculated from ISO sensitivity, ashutter speed, and an aperture value. As an imaging parameter fordetermining a scene, an imaging parameter of one type may be used orimaging parameters of a plurality of types may be used.

When there are a plurality of parameters for determining a scene, thescene determining unit 1105 may determine a plurality of scenes using aparameter in accordance with a set photography mode among the pluralityof parameters. For example, in the case of a photography mode in whichISO sensitivity is automatically set, a plurality of scenes may bedetermined on the basis of a time variation of the ISO sensitivity andnot on the basis of a time variation of parameters other than the ISOsensitivity. In the case of a photography mode in which white balance isautomatically set, a plurality of scenes may be determined on the basisof a time variation of the white balance and not on the basis of a timevariation of parameters other than the white balance. In a similarmanner, an aperture value may be used in the case of an aperturepriority mode and a shutter speed may be used in the case of a shutterspeed priority mode. In the case of a manual mode in which all of theimaging parameters are to be manually set, a plurality of scenes may bedetermined in consideration of all of the imaging parameters. At leastone of a plurality of parameters may be designated by the user as aparameter for determining a scene.

Control may be performed so that the scene determination process is notperformed (a plurality of scenes are not determined) when a specificphotography mode is set. For example, control may be performed so as notto perform a scene change during a focus change in an AF photographymode since it is highly likely that an image of a same scene is beingphotographed and to perform a scene change during a focus change in anMF photography mode since intended image formation is being performed.Execution/non-execution of the scene determination process may bedesignated by the user regardless of the photography mode or the like.

A parameter for determining a scene, a method of switching betweenexecution and non-execution of the scene determination process, aphotography mode in which the scene determination process is to beexecuted (or not to be executed), and the like are not particularlylimited.

Third Embodiment

A third embodiment of the present invention will be described below.Hereinafter, points (configurations, processes, and the like) thatdiffer from those of the first embodiment will be described in detailand descriptions of points that are the same as those of the firstembodiment will be omitted. In the present embodiment, an example ofdetermining a scene and generating dynamic metadata in consideration ofan in-focus region (a region inside a depth of field) of a frame imagewill be described. While a photography mode is not particularly limited,in the present embodiment, an example of the MF photography mode will bedescribed.

FIG. 7 is a block diagram showing a configuration example of an imagingapparatus 1700 according to the present embodiment. In FIG. 7, sameblocks as those shown in FIG. 1 (the first embodiment) are assigned samereference characters as in FIG. 1. The imaging apparatus 1700 includes acharacteristic amount acquiring unit 1704 in place of the characteristicamount acquiring unit 1104 according to the first embodiment andincludes a scene determining unit 1705 in place of the scene determiningunit 1105 according to the first embodiment. In addition, the imagingapparatus 1700 includes a depth of field calculation unit 1715.

The characteristic amount acquiring unit 1704 has a similar function tothe characteristic amount acquiring unit 1104 according to the firstembodiment. However, the characteristic amount acquiring unit 1704acquires, as a frame maximum brightness value, a maximum brightnessvalue of image data corresponding to an in-focus region (a region insidea depth of field) of a frame image instead of a maximum brightness valueof an entire of frame image data. The in-focus region (the region insidea depth of field) is notified by the depth of field calculation unit1715. In a similar manner to the first embodiment, the frame maximumbrightness value (a maximum brightness value of image data correspondingto the in-focus region) is used as both a characteristic amount fordetermining a plurality of scenes and a characteristic amount foracquiring a scene characteristic amount. It should be noted that amaximum brightness value of image data corresponding to the in-focusregion may be acquired as a characteristic amount for determining aplurality of scenes and a maximum brightness value of an entire of frameimage data may be acquired as a characteristic amount for acquiring ascene characteristic amount. Alternatively, the opposite may apply.

The scene determining unit 1705 has a similar function to the scenedetermining unit 1105 according to the first embodiment. However, whenthe frame maximum brightness value varies in a variation amount that islarger than a threshold, the scene determining unit 1705 determineswhether or not to perform a scene change in consideration of a timevariation of the in-focus region (the region inside a depth of field).The in-focus region (the region inside a depth of field) is notified bythe depth of field calculation unit 1715.

The depth of field calculation unit 1715 acquires an aperture value, afocus value (a focus position), and a zoom value (a focal length) fromthe imaging control unit 1103. In other words, the imaging control unit1103 outputs an aperture value, a focus value, and a zoom value to thedepth of field calculation unit 1715. The depth of field calculationunit 1715 calculates a depth of field from the aperture value, the focusvalue, and the zoom value. In addition, the depth of field calculationunit 1715 notifies the characteristic amount acquiring unit 1704 and thescene determining unit 1705 of a region inside the depth of field of aframe image as the in-focus region.

An example of a processing flow related to the scene determinationprocess will be described. First, the operating unit 1114 receives amode change operation (a user operation) for changing to the MFphotography mode. In accordance with the mode change operation, the CPU1111 controls the respective blocks of the imaging apparatus 1700 andsets the MF photography mode. Next, the imaging control unit 1103controls a state of the imaging optical system 1101, processing of theimaging element 1102, or the like in accordance with an aperture value,a shutter speed, a focus value, a zoom value, and the like. In addition,the imaging control unit 1103 outputs the aperture value, the focusvalue, and the zoom value to the depth of field calculation unit 1715.

Subsequently, the depth of field calculation unit 1715 calculates adepth of field from the aperture value, the focus value, and the zoomvalue acquired from the imaging control unit 1103. For example, thedepth of field is calculated using Expression 1 below.

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 1} \right\rbrack & \; \\{{{Depth}\mspace{14mu}{of}\mspace{14mu}{field}} = \frac{{aperture}\mspace{14mu}{value} \times {focus}\mspace{14mu}{value}^{2}}{{{zoom}\mspace{14mu}{value}^{2}} \pm {{aperture}\mspace{14mu}{value} \times {focus}\mspace{14mu}{value}}}} & \left( {{Expression}\mspace{14mu} 1} \right)\end{matrix}$

Next, on the basis of frame image data acquired from the imaging element1102 and the calculated depth of field, the depth of field calculationunit 1715 detects a region in the depth of field of the frame imagerepresented by the frame image data. In addition, the depth of fieldcalculation unit 1715 notifies the characteristic amount acquiring unit1704 and the scene determining unit 1705 of the region inside the depthof field (the in-focus region). It should be noted that a method ofdetecting the in-focus region is not particularly limited. For example,an edge region having a predetermined spatial frequency band may bedetected (edge detection) and an image region in which a density of thedetected edge region is higher than a predetermined threshold may bedetermined as the in-focus region.

Next, the operating unit 1114 receives a photography start operation. Inaccordance with the photography start operation, the CPU 1111 controlsthe respective blocks of the imaging apparatus 1700 and startsphotography. In addition, the characteristic amount acquiring unit 1704acquires a maximum brightness value of image data corresponding to theregion in a depth of field notified from the depth of field calculationunit 1715 as the frame maximum brightness value, and outputs the framemaximum brightness value to the scene determining unit 1105. Next, thescene determining unit 1705 determines a plurality of scenes using theframe maximum brightness value acquired from the characteristic amountacquiring unit 1104 and the region in a depth of field notified from thedepth of field calculation unit 1715.

FIGS. 8A to 8D show an example of frame images. In FIGS. 8A to 8D, afocus value that brings an object 1800 into focus is set. In FIGS. 8Aand 8D, a deep depth of field is set and a region 1801 (an entire frameimage) is set as an in-focus region. In FIGS. 8B and 8C, a shallow depthof field is set, a region 1802 (a part of the frame image) which isnarrower than the region 1801 is set as an in-focus region, and a region1803 of the region 1801 excluding the region 1802 is set as anout-of-focus region.

In the present embodiment, a plurality of scenes are determined on thebasis of a time variation of a frame maximum brightness value of thein-focus region. Therefore, even if a high brightness region 1804 iscreated inside the out-of-focus region 1803 and the frame maximumbrightness value increases in an amount of increase that is larger thana threshold as shown in FIG. 8C, a scene is not determined (finalized)(a scene change is not performed). Accordingly, in display on the basisof dynamic metadata, a variation in an appearance of an object (a likelyobject of interest) inside the in-focus region due to a brightnessvariation that occurs outside of the in-focus region can be prevented.

In the present embodiment, when a state where the depth of field isshallow (a state where the in-focus region is narrow; FIG. 8C) changesto a state where the depth of field is deep (a state where the in-focusregion is wide; FIG. 8D), a scene is determined (finalized) (a scenechange is performed) in accordance with a large variation in framemaximum brightness. Accordingly, the object 180X) affected by the highbrightness region 1804 can be displayed. On the other hand, when thestate where the depth of field is deep (the state where the in-focusregion is wide; FIG. 8D) changes to the state where the depth of fieldis shallow (the state where the in-focus region is narrow; FIG. 8C), ascene is not determined (finalized) (a scene change is not performed)regardless of a time variation in the frame maximum brightness.Accordingly, the object 1800 can be displayed without varying anappearance of the object 1800.

As described above, according to the present embodiment, by taking anin-focus region into consideration, more suitable dynamic metadata canbe acquired and a plurality of scenes can be determined in a moresuitable manner.

Fourth Embodiment

A fourth embodiment of the present invention will be described below.FIG. 9 is a block diagram showing an example of a configuration of animage processing apparatus 2100 according to the present embodiment. Theimage processing apparatus 2100 includes a storage unit 2101, a metadataextracting unit 2102, a metadata generating unit 2103, a moving imagedata connecting unit 2104, a metadata adding unit 2105, a CPU 2106, aRAM 2107, a ROM 2108, and an operating unit 2109.

The storage unit 2101 is a random-access recording medium such as a CF(compact flash) card, and moving image data to which static metadata anddynamic metadata have been added is recorded in the storage unit 2101.The storage unit 2101 is detachable from the image processing apparatus2100 and can be mounted to an apparatus other than the image processingapparatus 2100 (such as an imaging apparatus). Alternatively, thestorage unit 2101 may be an internal recording medium that is notattachable to and detachable from the image processing apparatus 2100.

In the present embodiment, it is assumed that moving image data encodedby MPEG-4 AVC or HEVC (High Efficiency Video Coding) is recorded in thestorage unit 2101. It is assumed that static metadata is added as adescriptor in MPEG-4 AVC or HEVC. It is assumed that dynamic metadata isadded to SEI (Supplemental Enhancement Information) in MPEG-4 AVC orHEVC. It should be noted that a method of encoding moving image data, amethod of adding static metadata, and a method of adding dynamicmetadata are not particularly limited.

Static metadata is metadata indicating a brightness independent of aframe and a scene of a moving image and, in the present embodiment,static metadata is assumed to be metadata indicating a maximumbrightness of an entire corresponding moving image (an entire movingimage to which the static metadata has been added). Dynamic metadata ismetadata indicating a brightness that varies depending on a frame or ascene of a moving image and, in the present embodiment, dynamic metadatais assumed to be metadata indicating a maximum brightness of each frameor each scene of a corresponding moving image.

The metadata extracting unit 2102 acquires moving image data from thestorage unit 2101 in accordance with a user operation with respect tothe operating unit 2109 and extracts static metadata and dynamicmetadata from the acquired moving image data. The metadata extractingunit 2102 outputs the extracted static metadata and dynamic metadata tothe metadata generating unit 2103. In addition, the metadata extractingunit 2102 outputs the acquired moving image data to the moving imagedata connecting unit 2104. In the present embodiment, the metadataextracting unit 2102 applies a decoding process on the basis of MPEG-4AVC or HEVC to the acquired moving image data. In addition, the metadataextracting unit 2102 outputs the moving image data after the decodingprocess to the moving image data connecting unit 2104.

The metadata generating unit 2103 generates static metadata and dynamicmetadata to be added to connected moving image data (moving image datato be generated by the moving image data connecting unit 2104), andoutputs the generated static metadata and dynamic metadata to themetadata adding unit 2105. In the present embodiment, the metadatagenerating unit 2103 generates static metadata of the connected movingimage data on the basis of a plurality of static metadata (a pluralityof static metadata respectively added to a plurality of moving imagedata which are connection targets) output from the metadata extractingunit 2102. Accordingly, suitable metadata can be generated as the staticmetadata of the connected moving image data. In addition, in the presentembodiment, the metadata generating unit 2103 generates dynamic metadataof the connected moving image data by connecting a plurality of dynamicmetadata (a plurality of dynamic metadata respectively added to aplurality of moving image data which are connection targets) output fromthe metadata extracting unit 2102. Accordingly, suitable metadata can begenerated as the dynamic metadata of the connected moving image data.

The moving image data connecting unit 2104 connects the plurality ofmoving image data output from the metadata extracting unit 2102 togenerate connected moving image data that represents a connected movingimage in which a plurality of moving images respectively represented bythe plurality of moving image data are connected. In addition, themoving image data connecting unit 2104 outputs the connected movingimage data to the metadata adding unit 2105.

The metadata adding unit 2105 adds the static metadata and the dynamicmetadata output from the metadata generating unit 2103 to the connectedmoving image data output from the moving image data connecting unit2104, and records the connected moving image data after adding themetadata thereto in the storage unit 2101. In the present embodiment,the metadata adding unit 2105 applies an encoding process on the basisof MPEG-4 AVC or HEVC to the connected moving image data. In addition,the metadata adding unit 2105 adds the static metadata as a descriptorin MPEG-4 AVC or HEVC and adds the dynamic metadata to SEI in MPEG-4 AVCor HEVC.

The CPU 2106 is connected via an internal bus (not illustrated) to otherblocks of the image processing apparatus 2100. The CPU 2106 controlsprocessing of the image processing apparatus 2100. The RAM 2107 isconnected via the internal bus (not illustrated) to other blocks of theimage processing apparatus 2100. The RAM 2107 is used as a work area ofthe CPU 2106 or a temporary storage area for temporarily storing variousdata. The ROM 2108 is connected via the internal bus (not illustrated)to other blocks of the image processing apparatus 2100. Firmware relatedto processing by the CPU 2106, information related to processing by theCPU 2106, and the like are recorded in the ROM 2108 in advance.

The operating unit 2109 is connected to the CPU 2106 via the internalbus (not illustrated). The operating unit 2109 is constituted by variousoperating members as an input unit for receiving user operations. Inaddition, the operating unit 2109 includes a touch panel and a liquidcrystal panel (both not illustrated) and causes displayed function iconsto act as various function buttons. For example, the operating unit 2109can receive a user operation (a moving image data connection operation)which instructs a plurality of moving image data to be selected asconnection targets from a plurality of moving image data stored in thestorage unit 2101 and the plurality of selected moving image data to beconnected. In accordance with the moving image data connectionoperation, the metadata extracting unit 2102 acquires a plurality ofmoving image data (a plurality of moving image data selected by theuser) from the storage unit 2101.

FIG. 10 is a flow chart showing an example of a connected moving imagedata generation process (a process of generating and recording connectedmoving image data) of the image processing apparatus 2100. The connectedmoving image data generation process shown in FIG. 10 is started whenthe CPU 2106 detects that the moving image data connection operationdescribed above has been received by the operating unit 2109, andrealized by the CPU 2106 by controlling the respective blocks of theimage processing apparatus 2100.

In S2201, the metadata extracting unit 2102 reads a plurality of movingimage data selected by the moving image data connection operation fromthe storage unit 2101. In this case, it is assumed that the moving imagedata A and B shown in FIG. 11 have been read.

FIG. 11 is a schematic view showing the moving image data A and B whichrepresent an example of moving image data that is a connection target.The moving image data A is constituted by two scenes, namely, a scene A1and a scene A2. Dynamic metadata of the moving image data A indicates amaximum brightness (a scene maximum brightness) “400” of the scene A1and a scene maximum brightness “250” of the scene A2. Therefore, amaximum brightness (a moving image maximum brightness) of an entiremoving image represented by the moving image data A is a same brightnessas the scene maximum brightness “400” of the scene A1, and staticmetadata of the moving image data A indicates the moving image maximumbrightness “400” of the moving image data A. The moving image data B isconstituted by one scene B1. Therefore, dynamic metadata of the movingimage data B indicates a same brightness as a brightness indicated bystatic metadata of the moving image data B. Specifically, the dynamicmetadata of the moving image data B indicates a scene maximum brightness“500” of the scene B1 and the static metadata of the moving image data Bindicates a moving image maximum brightness “500” of the moving imagedata B.

In S2202, the metadata extracting unit 2102 extracts static metadata anddynamic metadata from the moving image data A and B acquired in S2201,and outputs the extracted static metadata and dynamic metadata to themetadata generating unit 2103. In addition, the metadata extracting unit2102 outputs the moving image data A and B to the moving image dataconnecting unit 2104.

In S2203, the metadata generating unit 2103 generates static metadataand dynamic metadata to be added to connected moving image data on thebasis of the plurality of static metadata and the plurality of dynamicmetadata output from the metadata extracting unit 2102. In addition, themetadata generating unit 2103 outputs the generated static metadata anddynamic metadata to the metadata adding unit 2105.

In the present embodiment, the metadata generating unit 2103 generates,as the static metadata of the connected moving image data, metadataindicating a maximum brightness among a plurality of brightnessesrespectively indicated by the plurality of static metadata output fromthe metadata extracting unit 2102. As described above, the staticmetadata of the moving image data A indicates the moving image maximumbrightness “400” of the moving image data A and the static metadata ofthe moving image data B indicates the moving image maximum brightness“500” of the moving image data B. Therefore, the metadata generatingunit 2103 generates metadata indicating the brightness “500” as thestatic metadata of the connected moving image data.

As described earlier, the metadata generating unit 2103 generatesdynamic metadata of connected moving image data by connecting theplurality of dynamic metadata output from the metadata extracting unit2102. Specifically, the metadata generating unit 2103 generates dynamicmetadata of connected moving image data by simply splicing together theplurality of dynamic metadata. In other words, the metadata generatingunit 2103 generates metadata sequentially indicating scenes A1, A2, andB1 such that the scene B1 is subsequent to the scene A2 as the dynamicmetadata of the connected moving image data. Even in the dynamicmetadata of the connected moving image data, the brightness “400” isindicated as a scene maximum brightness of the scene A1, the brightness“250” is indicated as a scene maximum brightness of the scene A2, andthe brightness “500” is indicated as a scene maximum brightness of thescene B1.

In S2204, the moving image data connecting unit 2104 connects the movingimage data A and B output from the metadata extracting unit 2102 togenerate connected moving image data C shown in FIG. 12. In addition,the moving image data connecting unit 2104 outputs the connected movingimage data to the metadata adding unit 2105.

FIG. 12 is a schematic view showing the connected moving image data Cgenerated by connecting the moving image data A and B. As shown in FIG.12, in the connected moving image data C, the scene B1 of the movingimage data B is subsequent to the scenes A1 and A2 of the moving imagedata A. In addition, even in the connected moving image data C, thescene maximum brightness of the scene A1 is the brightness “400”, thescene maximum brightness of the scene A2 is the brightness “250”, andthe scene maximum brightness of the scene B1 is the brightness “500”.This shows that, in S2203, metadata accurately indicating a scenemaximum brightness of each scene has been generated as the dynamicmetadata of the connected moving image data C.

Furthermore, as shown in FIG. 12, a moving image maximum brightness ofthe connected moving image data C in which the scene B1 of the movingimage data B is subsequent to the scenes A1 and A2 of the moving imagedata A is a same brightness as the scene maximum brightness “500” of thescene B1. This shows that, in S2203, metadata accurately indicating amoving image maximum brightness of the connected moving image data C hasbeen generated as the static metadata of the connected moving image dataC.

In S2205, the metadata adding unit 2105 adds the static metadata and thedynamic metadata generated in S2203 to the connected moving image data Cgenerated in S2204.

In S2206, the metadata adding unit 2105 records the connected movingimage data C after the addition in S2205 (the connected moving imagedata C after adding the static metadata and the dynamic metadatathereto) in the storage unit 2101.

As described above, according to the present embodiment, suitablemetadata can be generated and added with respect to connected movingimage data.

While an example in which static metadata and dynamic metadata aregenerated and added to connected moving image data has been described inthe present embodiment, static metadata may be generated and added tothe connected moving image data without performing the processes relatedto dynamic metadata. The processes related to dynamic metadata includethe process of extracting dynamic metadata of moving image data readfrom the storage unit 2101, the process of generating dynamic metadataof connected moving image data, and the process of adding the dynamicmetadata to the connected moving image data.

It should be noted that while an example in which two moving image dataA and B are connected has been described in the present embodiment,three or more moving image data may be considered connection targets.

Moreover, while an example in which static metadata of connected movingimage data is generated so as to indicate a maximum brightness of anentire connected moving image has been described in the presentembodiment, the static metadata of connected moving image data mayindicate other brightnesses. For example, metadata indicating an averagebrightness of a plurality of brightnesses respectively indicated by aplurality of static metadata (a plurality of static metadatarespectively added to a plurality of moving image data that areconnection targets) may be generated as the static metadata of theconnected moving image data. Accordingly, when displaying a connectedmoving image on a display surface at a display brightness (a brightnesson the display surface) in accordance with the static metadata of theconnected moving image data, a variation of the display brightness frombefore the connection can be reduced over the entire connected movingimage. Metadata indicating another representative brightness (a minimumbrightness, a modal brightness, an intermediate brightness, or the like)of a plurality of brightnesses may be generated as the static metadataof connected moving image data.

Fifth Embodiment

A fifth embodiment of the present invention will be described below.FIG. 13 is a block diagram showing a configuration example of an imagingapparatus 2500 according to the present embodiment. The imagingapparatus 2500 includes an imaging optical system 2501, an imagingelement 2502, an imaging control unit 2503, a characteristic amountacquiring unit 2504, a metadata generating unit 2505, a metadata addingunit 2506, a storage unit 2507, a metadata updating unit 2508, a CPU2509, a RAM 2510, a ROM 2511, and an operating unit 2512.

The imaging optical system 2501 forms an optical image representing anobject on the imaging element 2502 in a similar manner to the imagingoptical system 1101 according to the first embodiment.

The imaging element 2502 captures a moving image (an object image)representing the object and generates moving image data representing themoving image. In the present embodiment, present moving image datagenerated by the imaging element 2502 (moving image data of a target tobe recorded in the storage unit 2507) will be described as “targetdata”. Specifically, the imaging element 2502 performs a photoelectricconversion process and an AD conversion process (analog-digitalconversion process) in a similar manner to the imaging element 1102according to the first embodiment. Subsequently, the imaging element2502 outputs frame image data obtained by the AD conversion process tothe characteristic amount acquiring unit 2504 and the metadata addingunit 2506. By repeating these processes, the imaging element 2502sequentially outputs a plurality of frame image data respectivelycorresponding to a plurality of frames of a moving image.

The imaging control unit 2503 controls imaging conditions of the imagingapparatus 2500 in a similar manner to the imaging control unit 1103according to the first embodiment.

The characteristic amount acquiring unit 2504 acquires a framecharacteristic amount from frame image data in a similar manner to thecharacteristic amount acquiring unit 1104 according to the firstembodiment. In the present embodiment, the characteristic amountacquiring unit 2504 acquires a maximum brightness (a frame maximumbrightness) of an image of one frame represented by the frame image dataand outputs the acquired frame maximum brightness to the metadatagenerating unit 2505. In the present embodiment, a plurality of framemaximum brightnesses respectively corresponding to a plurality of framesof a moving image (a target moving image) represented by the target dataare sequentially output from the characteristic amount acquiring unit2504.

The metadata generating unit 2505 generates static metadata and dynamicmetadata with respect to the target data. In other words, the metadatagenerating unit 2505 generates static metadata and dynamic metadata tobe added to the target data. In the present embodiment, the metadatagenerating unit 2505 generates static metadata indicating a moving imagemaximum brightness of the target data and dynamic metadata indicating ascene maximum brightness of the target data on the basis of theplurality of frame maximum brightnesses sequentially output from thecharacteristic amount acquiring unit 2504. The metadata generating unit2505 outputs the generated static metadata and dynamic metadata to themetadata adding unit 2506.

The metadata adding unit 2506 adds the static metadata and the dynamicmetadata output from the metadata generating unit 2505 to the targetdata generated by the imaging element 2502 (the moving image data madeup of a plurality of frame image data sequentially output from theimaging element 2502). In addition, the metadata adding unit 2506 iscapable of associating the target data with other moving image datagenerated in the past by the imaging element 2502. In the presentembodiment, the metadata adding unit 2506 adds identificationinformation (an identifier) to the target data. When associating thetarget data with other moving image data, identification informationthat is the same as identification information added to the other movingimage data is added to the target data. When not associating the targetdata with other moving image data, identification information thatdiffers from the identification information added to the other movingimage data is added to the target data. The metadata adding unit 2506records the target data after adding the static metadata, the dynamicmetadata, and the identification information thereto in the storage unit2507.

It should be noted that a method of adding the identificationinformation is not particularly limited. For example, the identificationinformation may be added as a part of static metadata. Theidentification information may be added (described) as at least a partof a file name of the target data. For example, a character string inwhich a file number is described after a character string correspondingto the identification information may be used as a file name. Theidentification information may be automatically generated or generatedmanually in accordance with a user operation. Generation of theidentification information may be switched between automatic generationand manual generation in accordance with the photography mode set by theCPU 2509. A method of association is not limited to a method usingidentification information. For example, a management table that managesassociation of a plurality of moving image data may be used to associatea plurality of moving image data.

The storage unit 2507 is a recording medium similar to the storage unit1109 according to the first embodiment and stores the moving image dataoutput from the metadata adding unit 2506 (the moving image data afteradding the static metadata, the dynamic metadata, and the identificationinformation thereto).

When the target data is associated with other moving image data, themetadata updating unit 2508 updates at least one of the static metadataof the other moving image data and the static metadata of the targetdata on the basis of the static metadata (so that the static metadataindicate a same brightness). The dynamic metadata is not updated.Accordingly, with respect to moving image data expected to be connected(moving image data prior to connection), suitable static metadata anddynamic metadata (static metadata and dynamic metadata expected to beconnected) can be generated and added. In the present embodiment, whenthe target data is associated with other moving image data, the metadataupdating unit 2508 reads the moving image data from the storage unit2507 and updates static metadata of at least any of the plurality ofread moving image data. Whether or not such an update is to be performedmay be selected in accordance with the photography mode set by the CPU2509.

The CPU 2509, the RAM 2510, and the ROM 2511 are respectively configuredin a similar manner to the CPU 1111, the RAM 1112, and the ROM 1113according to the first embodiment.

The operating unit 2512 is configured in a similar manner to theoperating unit 1114 according to the first embodiment and includes amoving image photography mode selection button for setting a movingimage photography mode and the like. The CPU 2509 sets the photographymode or changes the set photography mode in accordance with a useroperation with respect to the moving image photography mode selectionbutton. In addition, the operating unit 2512 is capable of receiving auser operation for designating identification information (anidentification information designation operation). The metadata addingunit 2506 adds identification information in accordance with theidentification information designation operation (identificationinformation designated by the user) to the target data.

FIG. 14 is a flow chart showing an example of a photography process bythe imaging apparatus 2500.

In S2601, the characteristic amount acquiring unit 2504 startsacquisition of a frame maximum brightness L_(MAX_NOW) of frame imagedata output from the imaging element 2502. The characteristic amountacquiring unit 2504 outputs the frame maximum brightness L_(MAX_NOW) tothe metadata generating unit 2505. The frame maximum brightnessL_(MAX_NOW) represents a frame maximum brightness of frame image datagenerated at the present timing t_(NOW).

In S2602, the metadata generating unit 2505 generates/updates staticmetadata and dynamic metadata (a scene maximum brightness and a movingimage maximum brightness) of the target data on the basis of the framemaximum brightness L_(MAX_NOW) at the present timing t_(NOW). In thepresent embodiment, during photography of the target moving image, theframe maximum brightness L_(MAX_NOW) is updated at a frame rate of thetarget data and the process of S2602 is repeated. In S2602 for a firsttime, the metadata generating unit 2505 records the frame maximumbrightness L_(MAX_NOW) as the scene maximum brightness and the movingimage maximum brightness in the RAM 2510. In S2602 for second andsubsequent times, the metadata generating unit 2505 compares the framemaximum brightness L_(MAX_NOW) with the brightness recorded in the RAM2510 (recorded brightness; the scene maximum brightness and the movingimage maximum brightness). In addition, when the frame maximumbrightness L_(MAX_NOW) is higher than the recorded brightness, themetadata generating unit 2505 updates the recorded brightness with theframe maximum brightness L_(MAX_NOW).

In S2603, the CPU 2509 determines whether or not the operating unit 2512has received a photography end operation by the user. When it isdetermined that a photography end operation has not been performed(S2603: No), the process is returned to S2602, but when it is determinedthat a photography end operation has been performed (S2603: Yes), theprocess is advanced to S2604. When the process advances from S2603 toS2604, the metadata generating unit 2505 reads the static metadata andthe dynamic metadata (the scene maximum brightness and the moving imagemaximum brightness) of the target data from the RAM 2510 and outputs theread static metadata and dynamic metadata to the metadata adding unit2506.

In S2604, the metadata adding unit 2506 adds the static metadata and thedynamic metadata output from the metadata generating unit 2505 to targetdata (moving image data made up of a plurality of frame image datasequentially output from the imaging element 2502). Furthermore, themetadata adding unit 2506 adds identification information to the targetdata.

In S2605, the metadata adding unit 2506 records the target data afteradding the static metadata, the dynamic metadata, and the identificationinformation thereto in the storage unit 2507.

In S2606, the CPU 2509 determines whether or not there is moving imagedata (stored in the storage unit 2507) to which the same identificationinformation as the identification information of the target data hasbeen added. In other words, the CPU 2509 determines whether or not thetarget data has been associated with other moving image data. When it isdetermined that there is no moving image data to which the sameidentification information as the identification information of thetarget data has been added or, in other words, the target data has notbeen associated with other moving image data (S2606: No), thephotography process shown in FIG. 14 is ended. When it is determinedthat there is moving image data to which the same identificationinformation as the identification information of the target data hasbeen added or, in other words, the target data has been associated withother moving image data (S2606: Yes), the process is advanced to S2607.

In S2607, the metadata updating unit 2508 updates at least one of thestatic metadata of the other moving image data associated with thetarget data and the static metadata of the target data on the basis ofthe static metadata. In the present embodiment, when a brightness (afirst brightness) indicated by the static metadata of the other movingimage data is higher than a brightness (a second brightness) indicatedby the static metadata of the target data, the static metadata of thetarget data is updated so as to indicate the first brightness. When thefirst brightness is lower than the second brightness, the staticmetadata of the other moving image data is updated so as to indicate thesecond brightness. As a result, after the update, both the staticmetadata of the other moving image data and the static metadata of thetarget data indicate a moving image maximum brightness of connectedmoving image data that is generated by connecting the target data andthe other moving image data. When the first brightness is equal to thesecond brightness, neither the static metadata of the other moving imagedata nor the static metadata of the target data is updated.

While metadata with the same static metadata as the target data isgenerated and added as the dynamic metadata of the target data in thephotography process shown in FIG. 14, this is not restrictive. Forexample, generation and an update of dynamic metadata may not beperformed in S2602 and addition of dynamic metadata may not be performedin S2605. Instead, prior to S2602, the metadata adding unit 2506 may addthe frame maximum brightness L_(MAX_NOW) at the present timing t_(NOW)to frame image data at the present timing t_(NOW) as a part of dynamicmetadata. In this case, dynamic metadata indicating a frame maximumbrightness of each frame of a target moving image is to be generated andadded to the target data. Although a period is created in which dynamicmetadata is added to the target data but static metadata is not added tothe target data, eventually, target data to which both dynamic metadataand static metadata are added is to be obtained. A scene change may beautomatically or manually detected during photography of a target movingimage and dynamic metadata indicating a scene maximum brightness of eachscene of the target moving image may be generated and added to thetarget data.

A specific example of the photography process shown in FIG. 14 will bedescribed with reference to FIGS. 15A to 15G FIGS. 15A to 15G areschematic views showing an example of a time variation of a framemaximum brightness, a scene maximum brightness (dynamic metadata), amoving image maximum brightness (static metadata), and the like ofmoving image data. In the photography process shown in FIG. 14, movingimage data (target data) representing a moving image constituted by onescene is generated and recorded. Therefore, each of the moving imagedata D1 to D3 shown in FIGS. 15A to 15G represents a moving imageconstituted by one scene.

First, let us assume that photography of the moving image data D1 startsat a timing T0. FIG. 15A shows a state during photography of the movingimage data D1. Among a plurality of frame maximum brightnesses obtainedup to the present timing t_(NOW) (a plurality of frame maximumbrightnesses of the moving image data D1), the frame maximum brightnessL_(MAX_NOW) “580” of the present timing t_(NOW) is highest. Therefore,due to the process of S2602, the frame maximum brightness L_(MAX_NOW)“580” is set as a scene maximum brightness and a moving image maximumbrightness of the moving image data D1.

Next, let us assume that the photography of the moving image data D1ends at a timing T1. FIG. 15B shows a state at the timing T1 at whichthe photography of the moving image data D1 ends or, in other words, astate where the present timing t_(NOW) is the timing T1. A frame maximumbrightness that is higher than the frame maximum brightness “580” hasbeen obtained prior to the timing T1 and, among a plurality of framemaximum brightnesses obtained up to the timing T1 (a plurality of framemaximum brightnesses of the moving image data D1), a frame maximumbrightness “600” is highest. Therefore, due to the process of S2602, thescene maximum brightness and the moving image maximum brightness of themoving image data D1 are updated from the brightness “580” to thebrightness “600”.

Subsequently, dynamic metadata indicating the scene maximum brightness“600” and static metadata indicating the moving image maximum brightness“600” are added to the moving image data D1 (S2604). The moving imagedata D1 after adding the static metadata and the dynamic metadatathereto is recorded in the storage unit 2507 (S2605). At this point, letus assume that the moving image data D1 has not been associated withother moving image data. Therefore, the process of S2607 is notperformed and the static metadata of the moving image data D1 is notupdated.

Next, let us assume that photography of the moving image data D2 startsat a timing T2. FIG. 15C shows a state at the timing T2 at which thephotography of the moving image data D2 starts or, in other words, astate where the present timing t_(NOW) is the timing T2.

In addition, let us assume that the photography of the moving image dataD2 ends at a timing T3. FIG. 15D shows a state at the timing T3 at whichthe photography of the moving image data D2 ends or, in other words, astate where the present timing t_(NOW) is the timing T3. Among aplurality of frame maximum brightnesses obtained from the timing T2 tothe timing T3 (the present timing t_(NOW)) (a plurality of frame maximumbrightnesses of the moving image data D2), a frame maximum brightness“550” is highest. Therefore, due to the process of S2602, the brightness“550” is set as a scene maximum brightness and a moving image maximumbrightness of the moving image data D2.

Subsequently, dynamic metadata indicating the scene maximum brightness“550” and static metadata indicating the moving image maximum brightness“550” are added to the moving image data D2 (S2604). The moving imagedata D2 after adding the static metadata and the dynamic metadatathereto is recorded in the storage unit 2507 (S2605).

At this point, let us assume that the moving image data D2 has beenassociated with the moving image data D1. Therefore, due to the processof S2607, the static metadata of the moving image data D1 or the staticmetadata of the moving image data D2 is updated. Specifically, since themoving image maximum brightness “600” of the moving image data D1 ishigher than the moving image maximum brightness “550” of the movingimage data D2, the static metadata of the moving image data D2 isupdated so as to indicate the moving image maximum brightness “600” ofthe moving image data D1. The dynamic metadata of the moving image dataD2 is not updated and indicates the scene maximum brightness “550” ofthe moving image data D2. FIG. 15E shows a state after the staticmetadata of the moving image data D2 has been updated. It should benoted that, when the moving image maximum brightness of the moving imagedata D1 is lower than the moving image maximum brightness of the movingimage data D2, the static metadata of the moving image data D1 isupdated so as to indicate the moving image maximum brightness of themoving image data D2.

Next, let us assume that photography of the moving image data D3 startsat a timing T4 and the photography of the moving image data D3 ends at atiming T5. FIG. 15F shows a state at the timing T5 at which thephotography of the moving image data D3 ends or, in other words, a statewhere the present timing t_(NOW) is the timing T5. Among a plurality offrame maximum brightnesses obtained from the timing T4 to the timing T5(the present timing t_(NOW)) (a plurality of frame maximum brightnessesof the moving image data D3), a frame maximum brightness “1000” ishighest. Therefore, due to the process of S2602, the brightness “1000”is set as a scene maximum brightness and a moving image maximumbrightness of the moving image data D3.

Subsequently, dynamic metadata indicating the scene maximum brightness“1000” and static metadata indicating the moving image maximumbrightness “1000” are added to the moving image data D3 (S2604). Themoving image data D3 after adding the static metadata and the dynamicmetadata thereto is recorded in the storage unit 2507 (S2605).

At this point, let us assume that the moving image data D3 has beenassociated with the moving image data D1 and D2. Therefore, due to theprocess of S2607, the static metadata of the moving image data D1 and D2or the static metadata of the moving image data D3 is updated.Specifically, a common brightness (a moving image maximum brightnessafter update shown in FIG. 15E) “600” respectively indicated by thestatic metadata of the moving image data D1 and the static metadata ofthe moving image data D2 is lower than the moving image maximumbrightness “1000” of the moving image data D3. Therefore, the staticmetadata of the moving image data D1 and D2 are updated so as toindicate the moving image maximum brightness “1000” of the moving imagedata D3. The dynamic metadata of the moving image data D1 and D2 are notupdated, the dynamic metadata of the moving image data D1 indicates thescene maximum brightness “600” of the moving image data D1, and thedynamic metadata of the moving image data D2 indicates the scene maximumbrightness “550” of the moving image data D2. FIG. 15G shows a stateafter the static metadata of the moving image data D1 and D2 have beenupdated. It should be noted that, when the moving image maximumbrightness (the common brightness described above) of the moving imagedata D1 and D2 is lower than the moving image maximum brightness of themoving image data D3, the static metadata of the moving image data D3 isupdated so as to indicate the moving image maximum brightness of themoving image data D1 and D2.

As shown in FIGS. 15A to 15G, in the present embodiment, metadataindicating a moving image maximum brightness of connected moving imagedata can be obtained as static metadata to be added to moving image dataprior to connection. In addition, as dynamic metadata to be added tomoving image data prior to connection, metadata indicating a scenemaximum brightness of the moving image data or the connected movingimage data can be obtained. As a result, connected moving image data towhich suitable static metadata and dynamic metadata has been added canbe readily generated. As an example, let us consider a simpleconfiguration in which any of a plurality of static metadatarespectively added to a plurality of moving image data prior toconnection is selected as static metadata of connected moving imagedata. Such a configuration enables static metadata indicating a movingimage maximum brightness of the connected moving image data to be alwaysselected.

As described above, according to the present embodiment, suitablemetadata can be generated and added with respect to moving image dataprior to connection.

It should be noted that static metadata may be generated and added tomoving image data without performing the processes related to dynamicmetadata (processes of generating dynamic metadata and adding thedynamic metadata to moving image data) in a similar manner to the fourthembodiment.

Depending on a method of association, when associating target data witha plurality of other moving image data, a brightness indicated by staticmetadata may differ among the plurality of other moving image data. Inthis case, for example, at least any of the plurality of static metadataof the target data and the plurality of other moving image data may beupdated so as to indicate a maximum brightness among a plurality ofbrightnesses indicated by the plurality of static metadata.

While an example in which static metadata of target data is updatedafter recording the target data to which the static metadata had beenadded is described in the present embodiment, a timing of updating thestatic metadata of the target data is not limited thereto. The staticmetadata added to the target data may be updated before recording thetarget data or the static metadata to be added to the target data may beupdated before adding the static metadata to the target data.

Moreover, while an example in which static metadata is updated to as toindicate a maximum brightness among a plurality of brightnessesrespectively indicated by a plurality of static metadata is described inthe present embodiment, the static metadata after the update mayindicate another brightness. For example, static metadata may be updatedso as to indicate another representative brightness (a minimumbrightness, a modal brightness, an intermediate brightness, an averagebrightness, or the like) of a plurality of brightnesses.

According to the fourth and fifth embodiments, the following aspects(configurations) are disclosed.

First Aspect

An image processing apparatus comprising:

a connecting unit configured to generate, by connecting a plurality ofmoving image data, connected moving image data representing a connectedmoving image in which a plurality of moving images respectivelyrepresented by the plurality of moving image data are connected;

a generating unit configured to generate metadata; and

an adding unit configured to add the metadata to the connected movingimage data, wherein

metadata indicating a maximum brightness of the entire of acorresponding moving image is added to each of the plurality of movingimage data as first metadata indicating a brightness independent of aframe and a scene of a moving image, and

the generating unit generates, on a basis of a plurality of firstmetadata respectively added to the plurality of moving image data, firstmetadata to be added to the connected moving image data.

Second Aspect

The image processing apparatus according to the first aspect, wherein

the generating unit generates, as the first metadata to be added to theconnected moving image data, metadata indicating a maximum brightnessamong a plurality of brightnesses respectively indicated by theplurality of first metadata.

Third Aspect

An image processing apparatus according to the first or second aspect,wherein

metadata indicating a maximum brightness of each frame or each scene ofthe corresponding moving image is further added to each of the pluralityof moving image data as second metadata indicating a brightness thatvaries depending on a frame or a scene of a moving image, and

the generating unit further generates, by connecting a plurality ofsecond metadata respectively added to the plurality of moving imagedata, second metadata to be added to the connected moving image data.

Fourth Aspect

An imaging apparatus comprising:

an image sensor configured to generate, by capturing a moving image,target data that is moving image data representing the moving image;

an associating unit capable of associating the target data with othermoving image data generated by the image sensor;

a generating unit configured to generate, with respect to the targetdata, metadata indicating a maximum brightness of the entire of a movingimage represented by the target data as first metadata indicating abrightness independent of a frame and a scene of a moving image;

an adding unit configured to add, to the target data, the first metadatagenerated with respect to the target data and

an updating unit configured to update, in a case where the target datais associated with the other moving image data, on a basis of the firstmetadata added to the other moving image data and the first metadata tobe added to or already added to the target data, at least one of thefirst metadata of the other moving image data and the first metadata ofthe target data.

Fifth Aspect

The imaging apparatus according to the fourth aspect, wherein

in a case where the target data is associated with the other movingimage data, the updating unit updates at least one of the first metadataof the other moving image data and the first metadata of the target dataso that the first metadata of the other moving image data and the firstmetadata of the target data indicate a same brightness.

Sixth Aspect

The imaging apparatus according to the fourth or fifth aspect, wherein

the associating unit adds identification information to the target data,and

the target data is associated with the other moving image data byaddition to the target data of identification information that is thesame as identification information added to the other moving image data.

Seventh Aspect

The imaging apparatus according to any one of the fourth to sixthaspects, wherein

in a case where the target data is associated with the other movingimage data and, and a first brightness indicated by the first metadataof the other moving image data is higher than a second brightnessindicated by the first metadata of the target data, the updating unitupdates the first metadata of the target data so as to indicate thefirst brightness.

Eighth Aspect

The imaging apparatus according to any one of the fourth to seventhaspects, wherein

in a case where the target data is associated with the other movingimage data and, and a first brightness indicated by the first metadataof the other moving image data is lower than a second brightnessindicated by the first metadata of the target data, the updating unitupdates the first metadata of the other moving image data so as toindicate the second brightness.

Ninth Aspect

The imaging apparatus according to any one of the fourth to eighthaspects, wherein

in a case where the target data is associated with a plurality of othermoving image data, the updating unit updates at least one of a pluralityof first metadata of the target data and the plurality of other movingimage data so as to indicate a maximum brightness among a plurality ofbrightnesses indicated by the plurality of first metadata.

Tenth Aspect

The imaging apparatus according to any one of the fourth to ninthaspects, wherein

if second moving image data representing a second moving image isgenerated and the second moving image data is associated with firstmoving image data representing a first moving image to which firstmetadata indicating a maximum brightness of the entire of the firstmoving image has been added,

the updating unit

-   -   updates first metadata of the second moving image data so as to        indicate the maximum brightness of the entire of the first        moving image in a case where the maximum brightness of the        entire of the first moving image is higher than a maximum        brightness of the entire of the second moving image, and    -   updates the first metadata of the first moving image data so as        to indicate the maximum brightness of the entire of the second        moving image in a case where the maximum brightness of the        entire of the first moving image is lower than the maximum        brightness of the entire of the second moving image.        Eleventh Aspect

The imaging apparatus according to the tenth aspect, wherein

if third moving image data representing a third moving image isgenerated and the third moving image data is associated with the firstmoving image data and the second moving image data which have beenassociated with each other,

the updating unit

-   -   updates first metadata of the third moving image data so as to        indicate a common brightness respectively indicated by the first        metadata of the first moving image data and the first metadata        of the second moving image data in a case where the common        brightness is higher than a maximum brightness of the entire of        the third moving image, and    -   updates the first metadata of the first moving image data and        the first metadata of the second moving image data so as to        indicate the maximum brightness of the entire of the third        moving image in a case where the common brightness is lower than        the maximum brightness of the entire of the third moving image.        Twelfth Aspect

The imaging apparatus according to any one of the fourth to eleventhaspects, further comprising

a setting unit configured to set one of a plurality of photographymodes, wherein

the updating unit switches between updating and not updating the firstmetadata in a case where the target data is associated with the othermoving image data in accordance with a photography mode set by thesetting unit.

Thirteenth Aspect

The imaging apparatus according to any one of the fourth to twelfthaspects, wherein

the generating unit further generates, with respect to the target data,metadata indicating a maximum brightness of each frame or each scene ofa moving image represented by the target data as second metadataindicating a brightness that varies depending on a frame or a scene of amoving image, and

the adding unit adds, to the target data, the second metadata generatedwith respect to the target data.

Sixth Embodiment

A sixth embodiment of the present invention will be described below.FIG. 16 is a block diagram showing a configuration example of an imagingapparatus 3100 according to the present embodiment. The imagingapparatus 3100 includes an imaging optical system 3101, an imagingelement 3102, an imaging control unit 3103, a characteristic amountacquiring unit 3104, a metadata generating unit 3105, a metadata addingunit 3106, a storage unit 3107, a reproducing unit 3108, a photographymode setting unit 3109, an output IF 3110, a CPU 3111, a RAM 3112, a ROM3113, and an operating unit 3114. In the present embodiment, “imaging”or “capturing” refers to a process of generating image data of an image(a still image or a moving image) representing an object, and“photography” refers to a process of recording image data generated byimaging in the storage unit 3107.

The imaging optical system 3101 forms an optical image representing anobject on the imaging element 3102 in a similar manner to the imagingoptical system 1101 according to the first embodiment.

The imaging element 3102 captures a still image (an object image)representing the object and generates still image data representing thestill image or captures a moving image (an object image) representingthe object and generates moving image data representing the movingimage. Specifically, the imaging element 3102 performs a photoelectricconversion process and an AD conversion process (analog-digitalconversion process) in a similar manner to the imaging element 1102according to the first embodiment. Subsequently, the imaging element3102 outputs frame image data obtained by the AD conversion process tothe characteristic amount acquiring unit 3104 and the metadata addingunit 3106. By repeating these processes, the imaging element 3102sequentially outputs a plurality of frame image data respectivelycorresponding to a plurality of frames of a moving image.

The imaging control unit 3103 controls imaging conditions of the imagingapparatus 3100 in a similar manner to the imaging control unit 1103according to the first embodiment.

The characteristic amount acquiring unit 3104 acquires a framecharacteristic amount from frame image data in a similar manner to thecharacteristic amount acquiring unit 1104 according to the firstembodiment. In the present embodiment, the characteristic amountacquiring unit 3104 acquires a frame maximum brightness and notifies themetadata generating unit 3105 of the acquired frame maximum brightness.In the present embodiment, the characteristic amount acquiring unit 3104sequentially notifies a plurality of frame maximum brightnessesrespectively corresponding to a plurality of frames of a captured movingimage.

The metadata generating unit 3105 generates dynamic metadata indicatinga characteristic amount of a captured moving image (a moving image to bephotographed). In the present embodiment, the metadata generating unit3105 generates dynamic metadata indicating, with respect to one or morescenes, a scene maximum brightness (a maximum brightness of one scene)of the captured moving image on the basis of the plurality of framemaximum brightnesses sequentially notified from the characteristicamount acquiring unit 3104. Alternatively, the metadata generating unit3105 generates dynamic metadata indicating a frame maximum brightness ofeach frame of the captured moving image on the basis of the plurality offrame maximum brightnesses sequentially notified from the characteristicamount acquiring unit 3104. The metadata generating unit 3105 outputsthe generated dynamic metadata to the metadata adding unit 3106.

While an example in which a characteristic amount indicated by dynamicmetadata is a maximum brightness such as a frame maximum brightness or ascene maximum brightness will be described in the present embodiment,the characteristic amount is not limited to a maximum brightness. Forexample, the characteristic amount indicated by dynamic metadata may beanother representative brightness (an average brightness, a minimumbrightness, a modal brightness, an intermediate brightness, or the like)of one frame, one scene, or the like. The characteristic amountindicated by dynamic metadata may be a brightness histogram of oneframe, one scene, or the like. The characteristic amount indicated bydynamic metadata may be a characteristic amount related to a color ofone frame, one scene, or the like. The characteristic amount indicatedby dynamic metadata may be a characteristic amount related to both abrightness and a color of one frame, one scene, or the like.

The metadata adding unit 3106 adds the dynamic metadata output from themetadata generating unit 3105 to the moving image data representing acaptured moving image (a moving image to be photographed). The movingimage data representing the captured moving image (the moving image tobe photographed) is moving image data made up of a plurality of frameimage data sequentially output from the imaging element 3102. Forexample, the metadata adding unit 3106 applies an encoding process onthe basis of MPEG-4 AVC or HEVC (High Efficiency Video Coding) to themoving image data. In addition, the metadata adding unit 3106 adds thedynamic metadata to SEI (Supplemental Enhancement Information) in MPEG-4AVC or HEVC. The metadata adding unit 3106 records the moving image dataafter adding the dynamic metadata thereto in the storage unit 3107.

The storage unit 3107 is a recording medium similar to the storage unit1109 according to the first embodiment and stores the moving image dataoutput from the metadata adding unit 3106 (the moving image data afteradding the dynamic metadata thereto).

The reproducing unit 3108 reproduces the moving image data stored in thestorage unit 3107. In the present embodiment, “reproduction” refers to aprocess of reading moving image data from the storage unit 3107 andoutputting the read moving image data to outside of the imagingapparatus 3100 via the output IF 3110.

The photography mode setting unit 3109 sets a photography mode. In thepresent embodiment, the photography mode setting unit 3109 is connectedvia an internal bus (not illustrated) to other functional blocks of theimaging apparatus 3100. The photography mode setting unit 3109 changessetting values, processes, and the like of the other functional blocksof the imaging apparatus 3100 in accordance with the set photographymode. In the present embodiment, the photography mode setting unit 3109changes a process of the metadata generating unit 3105 so that dynamicmetadata indicating a characteristic amount of a moving image in aperiod in accordance with the set photography mode is generated.Accordingly, suitable dynamic metadata can be generated. Details will beprovided later. For example, the photography mode setting unit 3109 setsa time-lapse photography mode, a continuous photography mode, a slowphotography mode, or a moving image simultaneous photography mode.

The time-lapse photography mode is a photography mode in which areproduction time of a moving image is shorter than a photography timeof the moving image. For example, the time-lapse photography mode is aphotography mode in which intermittent photography is performed and is aphotography mode for photographing a time-lapse moving image to bereproduced at a higher speed than actual time. In the time-lapsephotography mode, a moving image as though frames are decimated from amoving image photographed at an ordinary (standard) frame rate isphotographed. In a moving image photographed in the time-lapsephotography mode, a time variation of an object can be visually checkedin a short period of time.

The imaging apparatus 3100 can also photograph and reproduce a stillimage. For example, the imaging apparatus 3100 can capture a stillimage, add metadata to still image data representing the still image,and record the still image data in the storage unit 3107. The continuousphotography mode is a photography mode in which a plurality of stillimages are sequentially photographed. The reproducing unit 3108 cansequentially reproduce the plurality of still images photographed in thecontinuous photography mode (continuous reproduction).

The slow photography mode is a photography mode in which a reproductiontime of a moving image is longer than a photography time of the movingimage. For example, the slow photography mode is a photography mode inwhich photography is performed at a higher frame rate than an ordinaryframe rate and is a photography mode for photographing a slow movingimage to be reproduced at a lower speed than actual time.

The moving image simultaneous photography mode is a photography mode inwhich moving image data representing a moving image captured during apredetermined period is recorded linked with photography of a stillimage. For example, the moving image simultaneous photography mode is aphotography mode in which a moving image is captured so that the movingimage corresponding to a predetermined period of time up to the presenttiming is stored in the RAM 3112, and at a timing of photography of astill image, the moving image stored in the RAM 3112 is recorded in thestorage unit 3107 together with the still image. Using the moving imagesimultaneous photography mode enables not only a still image but also amoving image corresponding to a predetermined period of time up to atiming of photography of the still image to be checked. Alternatively, amoving image after the timing of photography of the still image may bemade checkable or a moving image corresponding to a period straddlingthe timing of photography of the still image may be made checkable.

The output IF 3110 outputs the moving image data stored in the storageunit 3107 (the moving image data after adding dynamic metadata thereto)to an external apparatus (not illustrated) in a similar manner to theoutput IF 1110 according to the first embodiment.

The CPU 3111, the RAM 3112, and the ROM 3113 are respectively configuredin a similar manner to the CPU 1111, the RAM 1112, and the ROM 1113according to the first embodiment.

The operating unit 3114 is configured in a similar manner to theoperating unit 1114 according to the first embodiment. Function buttons(function icons to act as function buttons) also include a photographymode selection button for selecting and setting a photography mode and areproduction start button for starting reproduction. The functionbuttons also include a setting button for designating and setting aphotography frame rate (a frame rate of photography). The functionbuttons also include a setting button for designating and setting areproduction frame rate (a frame rate of reproduction).

FIG. 17 is a flow chart showing an example of a photography process bythe imaging apparatus 3100. When the CPU 3111 detects a photographystart operation with respect to the operating unit 3114 in a state wherea special photography mode is set, the photography process shown in FIG.17 is started. Special photography mode is a general term for thetime-lapse photography mode, the continuous photography mode, the slowphotography mode, the moving image simultaneous photography mode, andthe like. Hereinafter, it is assumed that a change of the photographymode is not permitted during the photography process.

First, a case where photography is performed in the time-lapsephotography mode will be described. FIG. 18 is a schematic view showingan example of a relationship between a photography frame rate and areproduction frame rate in the time-lapse photography mode. Thephotography frame rate and the reproduction frame rate are set inaccordance with a user operation with respect to the operating unit3114. In the present embodiment, it is assumed that the photographyframe rate is 0.05 fps, the reproduction frame rate is 60 fps, and aphotography time is 20 hours. As shown in FIG. 18, from theseconditions, a reproduction time is determined as 1 minute (60seconds=72,000 seconds×0.05 fps/60 fps).

Before performing the photography start operation, with respect to theoperating unit 3114, the user performs a setting operation for settingthe time-lapse photography mode, a setting operation for setting thephotography frame rate, and a setting operation for setting thereproduction frame rate. In addition, settings in accordance with thesetting operations are made. For example, when the user performs asetting operation for designating 0.05 fps as the photography frame ratewith respect to the operating unit 3114, the CPU 3111 sets 0.05 fps asan imaging frame rate representing an imaging period at which theimaging element 3102 captures images. Furthermore, the CPU 3111 alsosets 0.05 fps as a recording frame rate representing a writing period atwhich the metadata adding unit 3106 writes frame image data into thestorage unit 3107. It should be noted that the imaging frame rate maydiffer from the recording frame rate. For example, an ordinary framerate such as 60 fps may be set as the imaging frame rate and 0.05 fpsmay be set as the recording frame rate. In this case, a captured movingimage is recorded in the storage unit 3107 after decimating framesthereof (the photography frame rate is 0.05 fps).

In the photography process shown in FIG. 17, first, the characteristicamount acquiring unit 3104 starts acquisition of a frame maximumbrightness of frame image data output from the imaging element 3102(S3201).

Next, the CPU 3111 determines whether or not the photography mode set tothe imaging apparatus 3100 is the time-lapse photography mode (S3202).When the photography mode is the time-lapse photography mode (S3202:Yes), the process is advanced to S3203, but when the photography mode isnot the time-lapse photography mode (S3202: No), the process is advancedto S3206. In this case, since it is assumed that the time-lapsephotography mode is set (S3202: Yes), the process is advanced to S3203.

In S3203, the metadata generating unit 3105 generates metadataindicating a frame maximum brightness of the present frame image data asthe metadata of the frame image data. Next, the metadata adding unit3106 adds the metadata output from the metadata generating unit 3105(the metadata generated in S3203) to the frame image data output fromthe imaging element 3102 (the present frame image data) (S3204). Inaddition, the metadata adding unit 3106 records the frame image dataafter adding the metadata thereto in the storage unit 3107. Theprocesses of S3203 and S3204 are to be repeated during photography.Therefore, in the case of the time-lapse photography mode, dynamicmetadata indicating a characteristic amount (a frame maximum brightness)of each frame of the captured moving image (the moving image to bephotographed) is generated and added to moving image data representingthe captured moving image (the moving image to be photographed). Itshould be noted that the metadata adding unit 3106 performs an encodingprocess of frame image data when appropriate as described above.

Subsequently, the CPU 3111 determines whether or not a photography endoperation with respect to the operating unit 3114 has been performed(S3205). When a photography end operation has not been performed (S3205:No), the process is returned to S3203. The processes of S3203 to S3205are repeated at the photography frame rate until a photography endoperation is performed. In other words, the processes of S3203 to S3205are performed for each frame of the captured moving image (the movingimage to be photographed). When a photography end operation has beenperformed (S3205: Yes), the photography process shown in FIG. 17 isended. In this case, it is assumed that the processes of S3203 to S3205had been repeated until 20 hours had elapsed from the start ofphotography (the start of the photography process shown in FIG. 17) andthat a photography end operation had been performed once 20 hours hadelapsed to end the photography process.

FIG. 19 is a schematic view showing an example of frame image data, aframe maximum brightness, and dynamic metadata of a moving imagephotographed in the time-lapse photography mode. Since the photographytime is 20 hours and the photography frame rate is 0.05 fps, 3600 (=0.05fps×72,000 seconds) frame images are photographed (frame image data no.1 to no. 3600). A solid line 3401 indicates a time variation of theframe maximum brightness. As shown in FIG. 19, when a moving image isphotographed in the time-lapse photography mode, metadata indicating acorresponding frame maximum brightness is added to each of the frameimage data no. 1 to no. 3600. For example, metadata indicating 100 nitsis added to the frame image data no. 1 to no. 3. In addition, metadataindicating 3000 nits is added to the frame image data no. M, andmetadata indicating 1000 nits is added to the frame image data no. 3600.In other words, dynamic metadata indicating a characteristic amount (aframe maximum brightness) of each frame of the captured moving image(the moving image to be photographed) is generated and added to movingimage data representing the captured moving image (the moving image tobe photographed).

While an example of setting the photography frame rate and thereproduction frame rate has been described, other parameters related toa frame rate (the photography frame rate, the reproduction frame rate,or the like) may be set. For example, a photography time (seconds) andthe number of frames to be photographed may be set. In this case, thephotography frame rate is obtained by dividing the number of frames tobe photographed by the photography time.

It should be noted that when the continuous photography mode is set or,more specifically, when continuous reproduction (a process ofsequentially reproducing a plurality of still images) is to be performedafter continuous photography (a process of sequentially photographing aplurality of still images), a photography process similar to when thetime-lapse photography mode is set may be applied. Specifically, themetadata generating unit 3105 may perform a process of generatingmetadata indicating a characteristic amount of a captured still imagefor each of a plurality of still images to be sequentially photographed.In addition, the metadata adding unit 3106 may perform a process ofadding metadata generated by the metadata generating unit 3105 to stillimage data representing the still image corresponding to the metadatafor each of a plurality of still images to be sequentially photographed.

In this case, for example, the still image data is recorded in EXIF(Exchangeable image file format). In addition, a characteristic amount(a maximum brightness) of a still image is described in metadataspecified in EXIF. These are processes performed during photography.Subsequently when sequentially reading a plurality of still image data(a plurality of still images) from the storage unit 3107, thereproducing unit 3108 extracts a characteristic amount (a maximumbrightness) from the metadata specified in EXIF and outputs thecharacteristic amount together with the still image data to the outputIF 3110. The output IF 3110 outputs the still image data output from thereproducing unit 3108 to an external apparatus in a stream formatcompliant with the HDMI standard. In other words, the still image datais output as a part (frame image data) of moving image data. At thispoint, the metadata (a characteristic amount; a maximum brightness)output from the reproducing unit 3108 is also output to the externalapparatus as a part (a frame maximum brightness) of dynamic metadataspecified in SMPTE ST 2094. These are processes performed duringreproduction.

According to the processes described above, even when performingcontinuous reproduction after continuous photography, dynamic metadataindicating a characteristic amount (a frame maximum brightness) of eachframe of a moving image can be generated and added to moving image data.

Next, a case where photography is performed in the slow photography modewill be described. FIG. 20 is a schematic view showing an example of arelationship between a photography frame rate and a reproduction framerate in the slow photography mode. The photography frame rate and thereproduction frame rate are set in accordance with a user operation withrespect to the operating unit 3114 in a similar manner to whenperforming photography in the time-lapse photography mode. In thepresent embodiment, it is assumed that the photography frame rate is 240fps, the reproduction frame rate is 60 fps, and a photography time is 1second. As shown in FIG. 20, from these conditions, a reproduction timeis determined as 4 seconds (=1 second×240 fps÷60 fps).

In the photography process shown in FIG. 17, first, processes of S3201and S3202 are performed in a similar manner to when performingphotography in the time-lapse photography mode. In other words, inS3201, the characteristic amount acquiring unit 3104 starts acquisitionof a frame maximum brightness of frame image data output from theimaging element 3102. In addition, in S3202, the CPU 3111 determineswhether or not the photography mode set to the imaging apparatus 3100 isthe time-lapse photography mode. In this case, since it is assumed thatthe time-lapse photography mode is not set but the slow photography modeis set instead (S3202: No), the process is advanced to S3206.

In S3206, the CPU 3111 determines whether or not the frame maximumbrightness of the present frame image data is higher than the framemaximum brightness (information) stored in the RAM 3112. In addition,when the CPU 3111 determines that the frame maximum brightness of thepresent frame image data is higher, the CPU 3111 updates the framemaximum brightness stored in the RAM 3112 with the frame maximumbrightness of the present frame image data. When the present frame imagedata corresponds to a first frame of a captured moving image (a movingimage to be photographed), the CPU 3111 records the frame maximumbrightness of the present frame image data in the RAM 3112.

In addition, the CPU 3111 determines whether or not a photography endoperation with respect to the operating unit 3114 has been performed(S3207). When a photography end operation has not been performed (S3207:No), the process is returned to S3206. The processes of S3206 and S3207are repeated at the photography frame rate until a photography endoperation is performed. In other words, the processes of S3206 and S3207are performed for each frame of the captured moving image (the movingimage to be photographed). When a photography end operation has beenperformed (S3207: Yes), the process is advanced to S3208. Therefore, theprocess is advanced to S3208 in a state where a characteristic amount (amaximum brightness) of the entire captured moving image (the movingimage to be photographed) is stored in the RAM 3112 or, in other words,a state where a scene maximum brightness when an entire period of amoving image is considered to be one scene is stored in the RAM 3112. Inthis case, it is assumed that the processes of S3206 and S3207 had beenrepeated until 1 second had elapsed from the start of photography (thestart of the photography process shown in FIG. 17) and that aphotography end operation had been performed once 1 second had elapsedto advance the process to S3208.

In S3208, the metadata generating unit 3105 generates, as dynamicmetadata, metadata indicating the frame maximum brightness stored in theRAM 3112 (a maximum brightness of an entire moving image; a scenemaximum brightness when an entire period of a moving image is consideredto be one scene).

Subsequently, the metadata adding unit 3106 adds the dynamic metadataoutput from the metadata generating unit 3105 (the dynamic metadatagenerated in S3208) to the moving image data output from the imagingelement 3102 (S3209). In addition, the metadata adding unit 3106 recordsthe moving image data after adding the dynamic metadata thereto in thestorage unit 3107. Subsequently, the photography process ends. Asdescribed earlier, the metadata adding unit 3106 performs an encodingprocess of the moving image data when appropriate as described above.

FIG. 21 is a schematic view showing an example of frame image data, aframe maximum brightness, and dynamic metadata of a moving imagephotographed in the slow photography mode. Since the photography time is1 second and the photography frame rate is 240 fps, 240 (=240 fps×1second) frame images are photographed (frame image data no. 1 to no.240). A solid line 3601 indicates a time variation of the frame maximumbrightness. As shown in FIG. 21, when a moving image is photographed inthe slow photography mode, unique dynamic metadata indicating a maximumbrightness among a plurality of frame maximum brightnesses respectivelycorresponding to the plurality of frame image data is added to movingimage data so as to correspond to all of the frame image data.Specifically, dynamic metadata indicating a frame maximum brightness3000 nits of frame image data no. N is added to moving image data asdynamic metadata corresponding to all of the frame image data no. 1 tono. 240. In other words, dynamic metadata indicating a characteristicamount (a maximum brightness) of the entire captured moving image (themoving image to be photographed) is generated and added to moving imagedata representing the captured moving image (the moving image to bephotographed).

It should be added that when the moving image simultaneous photographymode is set, a photography process similar to when the slow photographymode is set may be applied. However, in the case of the moving imagesimultaneous photography mode, in S3206, the CPU 3111 records presentframe image data in the RAM 3112 so that moving image data (a pluralityof frame image data) corresponding to a predetermined time up to thepresent timing is stored in the RAM 3112. Furthermore, the CPU 3111records a frame maximum brightness corresponding to the present frameimage data in the RAM 3112 so that a plurality of frame maximumbrightnesses respectively corresponding to the plurality of frame imagedata are stored in the RAM 3112. In S3207, the CPU 3111 determineswhether or not photography of a still image has been performed.Photography of a still image is performed in accordance with, forexample, a depression of a still image photography button. Subsequently,in S3208, the metadata generating unit 3105 generates, as dynamicmetadata, metadata indicating a maximum brightness among the pluralityof frame maximum brightnesses stored in the RAM 3112.

FIG. 22 is a schematic view showing an example of a tone curve (a tonemap) in a case where an upper limit of a display apparatus (a brightnessof a display surface) of a display apparatus according to the presentembodiment is set to 1000 nits. The tone curve represents a relationshipbetween a brightness (a data brightness) represented by image data inputto the display apparatus and a display brightness of the displayapparatus. The display apparatus displays an image in accordance withthe tone curve. In the present embodiment, frame image data output fromthe imaging apparatus 3100 (the output IF 3110) is input to the displayapparatus. The display apparatus changes the tone curve in accordancewith dynamic metadata output from the imaging apparatus 3100.

When using a tone curve 3701, while a data brightness 1000 nits isdisplayed at a display brightness 1000 nits, a data brightness that ishigher than 1000 nits is also displayed at the display brightness 1000nits. In other words, while a data brightness that is equal to or lowerthan 1000 nits can be faithfully displayed, a data brightness that ishigher than 1000 nits cannot be displayed with high gradation propertiesand gradation loss (blown-out highlights) occurs in an area of databrightness that is higher than 1000 nits. When using a tone curve 3702,while a decline in brightness occurs such as displaying a databrightness 1000 nits at a display brightness 800 nits, a data brightnessthat is higher than 1000 nits can be displayed with high gradationproperties.

As described earlier, when the time-lapse photography mode is set,dynamic metadata indicating a frame maximum brightness of each frame ofa moving image is generated. Therefore, in the display apparatus, a tonecurve is changed for each frame. For example, when displaying frameimage data no. M (frame maximum brightness 3000 nits) shown in FIG. 22,the tone curve 3702 is used. In addition, when displaying frame imagedata no. 3600 (frame maximum brightness 1000 nits) shown in FIG. 22, thetone curve 3701 is used. By changing the tone curve to be used from thetone curve 3702 to the tone curve 3701, the decline in brightnessdescribed above (in particular, a variation from a data brightness of100 nits or higher, a decline from a data brightness near 1000 nits, andthe like) can be suppressed. Furthermore, by changing the tone curve tobe used from the tone curve 3701 to the tone curve 3702, a databrightness that is higher than 1000 nits can be displayed with highgradation properties.

On the other hand, when the slow photography mode is set, dynamicmetadata indicating a scene maximum brightness when an entire period ofa moving image is considered to be one scene is generated. Therefore, inthe display apparatus, a tone curve is not changed. In the example shownin FIG. 21, since dynamic metadata indicating a scene maximum brightness3000 nits is generated, the tone curve 3702 is always used in thedisplay apparatus. As a result, a brightness variation attributable to avariation in the tone curve can be suppressed. In a moving imagephotographed in the slow photography mode, a difference in imagesbetween frames is relatively small. By suppressing a brightnessvariation attributable to a variation in the tone curve, an abruptvariation in an appearance of images between frames with a relativelysmall difference in the images can be suppressed.

As described above, according to the present embodiment, in an imagingapparatus, dynamic metadata is generated so as to indicate acharacteristic amount of a moving image in a period in accordance with aset photography mode and the generated dynamic metadata is added tomoving image data. Accordingly, moving image data to which suitabledynamic metadata has been added can be obtained in a photographingprocess (the imaging apparatus) instead of post-production (an imagedata editing process). For example, by using the dynamic metadata addedin the present embodiment, a time-lapse moving image can be displayedwhile suppressing a decline in brightness in a high brightness areahaving a brightness near a frame maximum brightness. A slow moving imagecan also be displayed while suppressing an abrupt variation inbrightness.

It should be noted that photography modes are not limited to thetime-lapse photography mode, the continuous photography mode, the slowphotography mode, and the moving image simultaneous photography mode.For example, a photography mode in which a photography frame rate can bechanged during photography may be made settable. When such a photographymode is set, the metadata generating unit 3105 may generate dynamicmetadata in which a period corresponding to a characteristic amountchanges at a timing where the photography frame rate is changed. Inother words, the metadata generating unit 3105 may generate dynamicmetadata indicating each scene maximum brightness while adopting atiming where the photography frame rate is changed as a timing of ascene change.

In addition, the metadata generating unit 3105 may generate dynamicmetadata so that, with respect to a period in which the photographyframe rate is lower than a predetermined frame rate, a characteristicamount (a frame maximum brightness) of each frame of a moving image isindicated instead of a scene maximum brightness. For example, thepredetermined frame rate is an ordinary frame rate such as 60 fps.

Furthermore, the metadata generating unit 3105 may generate dynamicmetadata so that, with respect to a period in which the photographyframe rate is higher than the predetermined frame rate, a periodcorresponding to the characteristic amount changes at a timing of ascene change of a moving image. In other words, the metadata generatingunit 3105 may generate dynamic metadata so that, with respect to aperiod in which the photography frame rate is higher than thepredetermined frame rate, each scene maximum brightness is indicated.Timings of a scene change may include a timing that differs from atiming where the photography frame rate has been changed. For example, atiming where the frame maximum brightness varies between frames by anamount equal to or larger than a threshold may be further used as atiming of a scene change.

Alternatively, the metadata generating unit 3105 may generate dynamicmetadata so that, with respect to a period in which the photographyframe rate is higher than the predetermined frame rate, a periodcorresponding to the characteristic amount changes at a predeterminedframe rate. In other words, the metadata generating unit 3105 maygenerate dynamic metadata indicating each scene maximum brightness whileadopting a timing where the photography frame rate has been changed anda timing of a predetermined frame rate as timings of a scene change.

Seventh Embodiment

A seventh embodiment of the present invention will be described below.In the sixth embodiment, an example has been described in which areproduction time (a reproduction frame rate) is set in advance. In theseventh embodiment, an example in which the reproduction time can bechanged when performing reproduction will be described. Hereinafter,points (configurations, processes, and the like) that differ from thoseof the sixth embodiment will be described in detail and descriptions ofpoints that are the same as those of the sixth embodiment will beomitted.

The photography mode setting unit 3109 is capable of setting a variableframe rate photography mode (a photography mode in which a photographyframe rate can be changed). When the variable frame rate mode is set, aswitch from ordinary photography to special photography, a switch fromspecial photography to another special photography, a switch fromspecial photography to ordinary photography, or the like is performed inaccordance with a user operation with respect to the operating unit3114. Special photography is the same as photography in the specialphotography mode, and examples include time-lapse photography that isphotography in the time-lapse photography mode and slow photography thatis photography in the slow photography mode.

In addition to the processes described in the sixth embodiment, thereproducing unit 3108 can also perform a scene determination process ofdetermining a scene of a moving image, a regeneration process ofregenerating (updating) dynamic metadata, and the like. In the scenedetermination process, for example, a timing where a frame maximumbrightness varies between frames by an amount equal to or larger than athreshold is detected as a timing of a scene change. Furthermore, thereproducing unit 3108 can set a reproduction mode in accordance with auser operation with respect to the operating unit 3114. In the presentembodiment, the reproducing unit 3108 sets a special effect prioritymode or a time priority mode when reproducing a moving imagephotographed in the variable frame rate photography mode. The specialeffect priority mode and the time priority mode will be described later.

A regeneration process of dynamic metadata in accordance with theregeneration mode of the imaging apparatus 3100 will be described. Inthis case, it is assumed that moving image data photographed in thevariable frame rate photography mode is to be reproduced and dynamicmetadata indicating a frame maximum brightness of each frame has beenadded to the moving image data to be reproduced.

First, a case where the special effect priority mode is set will bedescribed. The special effect priority mode is a reproduction mode inwhich reproduction prioritizing an effect of special photography isperformed. When the special effect priority mode is set, a moving imagephotographed in the variable frame rate photography mode is reproducedover a reproduction time that differs from a photography time of themoving image. For example, the moving image is reproduced over areproduction time that is longer than the photography time with respectto a period of slow photography (slow reproduction) but the moving imageis reproduced over a reproduction time that is shorter than thephotography time with respect to a period of time-lapse photography(time-lapse reproduction).

FIG. 23 is a schematic view showing an example of a relationship among aphotography frame rate, a reproduction frame rate, and a reproductionstate in a case where the special effect priority mode is set.

In FIG. 23, a moving image photographed in the variable frame ratephotography mode is made up of an ordinary photography moving image3801, a slow photography moving image 3802, a time-lapse photographymoving image 3803, and an ordinary photography moving image 3804. Theordinary photography moving images 3801 and 3804 are moving imagesobtained by ordinary photography and a photography frame rate of theordinary photography moving images 3801 and 3804 is 60 fps. The slowphotography moving image 3802 is a moving image obtained by slowphotography and a photography frame rate of the slow photography movingimage 3802 is 240 fps. The time-lapse photography moving image 3803 is amoving image obtained by time-lapse photography and a photography framerate of the time-lapse photography moving image 3803 is 30 fps.

When the special effect priority mode is set, ordinary reproduction ofthe ordinary photography moving image 3801 is performed, slowreproduction of the slow photography moving image 3802 is performed,time-lapse reproduction of the time-lapse photography moving image 3803is performed, and ordinary reproduction of the ordinary photographymoving image 3804 is performed. In the example shown in FIG. 23, areproduction moving image (a moving image that is reproduced) is made upof an ordinary reproduction moving image 3811, a slow reproductionmoving image 3812, a time-lapse reproduction moving image 3813, and anordinary reproduction moving image 3814. The ordinary reproductionmoving image 3811 corresponds to the ordinary photography moving image3801, the slow reproduction moving image 3812 corresponds to the slowphotography moving image 3802, the time-lapse reproduction moving image3813 corresponds to the time-lapse photography moving image 3803, andthe ordinary reproduction moving image 3814 corresponds to the ordinaryphotography moving image 3804. In addition, the reproduction frame rateis always set to 60 fps.

FIG. 24 is a schematic view showing an example of regenerated dynamicmetadata in a case where the special effect priority mode is set. Inthis case, a difference between a frame maximum brightness La and aframe maximum brightness Lb, a difference between a frame maximumbrightness Ld and a frame maximum brightness Le, and a differencebetween a frame maximum brightness Lg and a frame maximum brightness Lhare equal to or larger than the threshold described earlier fordetecting a scene change. A case where the special effect priority modeis set is a case where a moving image photographed in the variable framerate photography mode is reproduced over a reproduction time thatdiffers from a photography time of the moving image.

When the special effect priority mode is set, the reproducing unit 3108regenerates dynamic metadata so as to satisfy the following conditions.In this case, a timing where a scene of a moving image changes is not atiming where the photography frame rate has been changed but, forexample, a timing where the frame maximum brightness varies by an amountequal to or larger than a threshold. A timing where the photographyframe rate has been changed is a timing where a reproduction statechanges.

-   -   In a period in which the photography frame rate is a        predetermined frame rate (for example, an ordinary frame rate        such as 60 fps), a period corresponding to a characteristic        amount is changed at a timing where a scene of a moving image        changes and at a timing where the photography frame rate has        been changed.    -   In a period in which the photography frame rate is not a        predetermined frame rate, a period corresponding to a        characteristic amount is changed at a timing where the        photography frame rate has been changed.

In the present embodiment, with respect to a period of ordinaryreproduction, dynamic metadata is regenerated so as to indicate a scenemaximum brightness with a timing where the photography frame rate hasbeen changed and a timing where the frame maximum brightness varies byan amount equal to or larger than a threshold as timings of scenechange. Specifically, among a period of the ordinary reproduction movingimage 3811, a period from a time point 0 where reproduction is startedto a time point t1 where the frame maximum brightness varies from abrightness La to a brightness Lb is detected as a period of one scene.In addition, as a part of dynamic metadata to be regenerated or, morespecifically, metadata corresponding to the detected scene (the periodfrom the time point 0 to the time point t1), metadata indicating thescene maximum brightness La of the scene is regenerated. In a similarmanner, a period from the time point t1 to a time point t2 where areproduction state varies is detected as one scene and metadataindicating a scene maximum brightness Lc of the scene is regenerated asmetadata corresponding to the detected scene. With respect to a periodof the ordinary reproduction moving image 3814, a period from a timepoint t6 where the reproduction state varies to a time point t7 wherereproduction is ended is detected as a period of one scene. In addition,as metadata corresponding to the detected scene (the period from thetime point t6 to the time point t7), metadata indicating a scene maximumbrightness Li of the scene is regenerated.

Furthermore, with respect to a period of special reproduction (slowreproduction, time-lapse reproduction, or the like), dynamic metadata isgenerated so as to indicate a characteristic amount of an entire movingimage in the period. In other words, with respect to a period of specialreproduction, dynamic metadata is regenerated so as to indicate a scenemaximum brightness when the period is considered to be one scene.

Specifically, with respect to a period of the slow reproduction movingimage 3812, a period from the time point t2 where slow reproduction isstarted to a time point t4 where a reproduction state varies is detectedas one scene. In addition, as a part of dynamic metadata to beregenerated or, more specifically, metadata corresponding to thedetected scene (the period from the time point t2 to the time point t4),metadata indicating a scene maximum brightness Lf of the scene isregenerated. While metadata is changed at a time point t3 where theframe maximum brightness varies from a brightness Ld to a brightness Lein the case of ordinary reproduction, metadata is not changed at thetime point t3 in the case of slow reproduction.

In a similar manner, with respect to a period of the time-lapsereproduction moving image 3813, a period from the time point t4 wheretime-lapse reproduction is started to the time point t6 where areproduction state varies is detected as one scene. In addition, asmetadata corresponding to the detected scene (the period from the timepoint t4 to the time point t6), metadata indicating a scene maximumbrightness Lg of the scene is regenerated. While metadata is changed ata time point t5 where the frame maximum brightness varies from thebrightness Lg to a brightness Lh in the case of ordinary reproduction,metadata is not changed at the time point t5 in the case of time-lapsereproduction.

By using dynamic metadata regenerated by the process described above,since a tone curve is not changed during special reproduction, (anabrupt) brightness variation attributable to a variation in the tonecurve can be suppressed.

Next, a case where the time priority mode is set will be described. Thetime priority mode is a reproduction mode in which reproductionprioritizing photography time is performed. When the time priority modeis set, a moving image photographed in the variable frame ratephotography mode is reproduced over a reproduction time that is equal toa photography time of the moving image.

FIG. 25 is a schematic view showing an example of a relationship among aphotography frame rate, a reproduction frame rate, and a reproductionstate in a case where the time priority mode is set. In a similar mannerto FIG. 23, a moving image photographed in the variable frame ratephotography mode is made up of the ordinary photography moving image3801, the slow photography moving image 3802, the time-lapse photographymoving image 3803, and the ordinary photography moving image 3804.

When the time priority mode is set, ordinary reproduction of theordinary photography moving image 3801 is performed, decimatedreproduction of the slow photography moving image 3802 is performed,duplicate reproduction of the time-lapse photography moving image 3803is performed, and ordinary reproduction of the ordinary photographymoving image 3804 is performed. In the decimated reproduction, framesare decimated so that the reproduction time equals the photography time.In the duplicate reproduction, same frames are reproduced a plurality oftimes so that the reproduction time equals the photography time. In theexample shown in FIG. 25, a reproduction moving image is made up of anordinary reproduction moving image 3911, a decimated reproduction movingimage 3912, a duplicate reproduction moving image 3913, and an ordinaryreproduction moving image 3914. The ordinary reproduction moving image3911 corresponds to the ordinary photography moving image 3801, thedecimated reproduction moving image 3912 corresponds to the slowphotography moving image 3802, the duplicate reproduction moving image3913 corresponds to the time-lapse photography moving image 3803, andthe ordinary reproduction moving image 3914 corresponds to the ordinaryphotography moving image 3804. In addition, the reproduction frame rateis always set to 60 fps.

FIG. 26 is a schematic view showing an example of regenerated dynamicmetadata in a case where the time priority mode is set. In a similarmanner to FIG. 24, the difference between the frame maximum brightnessLa and the frame maximum brightness Lb, the difference between the framemaximum brightness Ld and the frame maximum brightness Le, and thedifference between the frame maximum brightness Lg and the frame maximumbrightness Lh are equal to or larger than the threshold describedearlier for detecting a scene change. A case where the time prioritymode is set is a case where a moving image photographed in the variableframe rate photography mode is reproduced over a reproduction time thatis equal to a photography time of the moving image.

When the time priority mode is set, the reproducing unit 3108regenerates dynamic metadata in which a period corresponding to acharacteristic amount changes at a timing where a scene of a movingimage is changed independent of the photography frame rate. In thiscase, a timing where a scene of a moving image changes is not a timingwhere the photography frame rate has been changed but, for example, atiming where the frame maximum brightness varies by an amount equal toor larger than a threshold. In the present embodiment, when the timepriority mode is set, a timing where the photography frame rate has beenchanged is not handled as a timing where a scene change occurs but atiming where the frame maximum brightness varies by an amount equal toor larger than a threshold is handled as a timing where a scene changeoccurs. In other words, dynamic metadata indicating a scene maximumbrightness is regenerated by handling the reproduction moving images3911 to 3914 as one reproduction moving image.

Specifically, a period from the time point 0 where reproduction isstarted to the time point t1 where the frame maximum brightness variesfrom the brightness La to the brightness Lb is detected as a period ofone scene. In addition, as a part of dynamic metadata to be regeneratedor, more specifically, metadata corresponding to the detected scene (theperiod from the time point 0 to the time point t1), metadata indicatingthe scene maximum brightness La of the scene is regenerated. In asimilar manner, a period from the time point t1 to the time point t3where the frame maximum brightness varies from the brightness Ld to thebrightness Le is detected as one scene and metadata indicating a scenemaximum brightness Ld of the scene is regenerated as metadatacorresponding to the detected scene. A period from the time point t3 tothe time point t5 where the frame maximum brightness varies from thebrightness Lg to the brightness Lh is detected as one scene and metadataindicating a scene maximum brightness Lg of the scene is regenerated asmetadata corresponding to the detected scene. In addition, a period fromthe time point t5 to the time point t7 where reproduction is ended isdetected as one scene and metadata indicating a scene maximum brightnessLh of the scene is regenerated as metadata corresponding to the detectedscene. In this manner, while metadata is changed at the time points t1,t3, and t5 where the frame maximum brightness varies by an amount equalto or larger than a threshold, metadata is not changed at the timepoints t2, t4, and t6 where the reproduction state varies.

By using dynamic metadata regenerated by the process described above,since a tone curve is not changed during reproduction of a scene of amoving image, a brightness variation attributable to a variation in thetone curve can be suppressed.

As described above, according to the present embodiment, dynamicmetadata is regenerated by a method in accordance with a reproductionmethod of a moving image photographed in the variable frame ratephotography mode. Accordingly, display at a suitable brightness can berealized using dynamic metadata suitable for a reproduction method.

According to the sixth and seventh embodiments, the following aspects(configurations) are disclosed.

First Aspect

An imaging apparatus comprising:

a setting unit configured to set a photography mode;

a generating unit configured to generate metadata indicating acharacteristic amount of a captured moving image; and

an adding unit configured to add the metadata generated by thegenerating unit to moving image data representing the moving image,wherein

the generating unit generates the metadata indicating the characteristicamount of the moving image in a period corresponding to a photographymode set by the setting unit.

Second Aspect

The imaging apparatus according to the first aspect, wherein

the characteristic amount is a maximum brightness.

Third Aspect

The imaging apparatus according to the first or second aspect, wherein

in a case where the photography mode set by the setting unit is aphotography mode in which a reproduction time of the moving image isshorter than a photography time of the moving image, the generating unitgenerates metadata indicating a characteristic amount of each frame ofthe moving image.

Fourth Aspect

The imaging apparatus according to any one of the first to thirdaspects, wherein

in a case where the photography mode set by the setting unit is aphotography mode in which a plurality of still images are sequentiallyphotographed,

-   -   the generating unit performs a process of generating metadata        indicating a characteristic amount of a captured still image        with respect to each of the plurality of still images that are        sequentially photographed, and    -   the adding unit performs a process of adding the metadata        generated by the generating unit to still image data        representing the still image corresponding to the metadata with        respect to each of the plurality of still images that are        sequentially photographed.        Fifth Aspect

The imaging apparatus according to any one of the first to fourthaspects, wherein

in a case where the photography mode set by the setting unit is aphotography mode in which a reproduction time of the moving image islonger than a photography time of the moving image, the generating unitgenerates metadata indicating a characteristic amount of the entire ofthe moving image.

Sixth Aspect

The imaging apparatus according to any one of the first to fifthaspects, wherein

in a case where the photography mode set by the setting unit is aphotography mode in which moving image data representing a moving imagecaptured during a predetermined period is recorded linked withphotography of a still image, the generating unit generates metadataindicating a characteristic amount of the entire of the moving imagecaptured during the predetermined period.

Seventh Aspect

The imaging apparatus according to any one of the first to sixthaspects, wherein

in a case where the photography mode set by the setting unit is aphotography mode in which a photography frame rate can be changed, thegenerating unit generates metadata in which the period corresponding tothe characteristic amount changes at a timing where the photographyframe rate is changed.

Eighth Aspect

The imaging apparatus according to the seventh aspect, wherein

in a case where the photography mode set by the setting unit is aphotography mode in which a photography frame rate can be changed, thegenerating unit generates metadata indicating a characteristic amount ofeach frame of the moving image with respect to a period in which thephotography frame rate is lower than a predetermined frame rate.

Ninth Aspect

The imaging apparatus according to the seventh or eighth aspect, wherein

in a case where the photography mode set by the setting unit is aphotography mode in which a photography frame rate can be changed, thegenerating unit generates metadata in which the period corresponding tothe characteristic amount changes at a timing where a scene of themoving image changes during a period in which the photography frame rateis higher than a predetermined frame rate.

Tenth Aspect

The imaging apparatus according to the seventh or eighth aspect, wherein

in a case where the photography mode set by the setting unit is aphotography mode in which a photography frame rate can be changed, thegenerating unit generates metadata in which the period corresponding tothe characteristic amount changes at a predetermined frame rate during aperiod in which the photography frame rate is higher than thepredetermined frame rate.

Eleventh Aspect

The imaging apparatus according to any one of the first to tenthaspects, wherein

in a case where a moving image photographed in a photography mode inwhich a photography frame rate can be changed is reproduced over areproduction time that is equal to a photography time of the movingimage, the generating unit regenerates metadata in which the periodcorresponding to the characteristic amount changes at a timing where ascene of the moving image is changed.

Twelfth Aspect

The imaging apparatus according to any one of the first to eleventhaspects, wherein

in a case where a moving image photographed in a photography mode inwhich a photography frame rate can be changed is reproduced over areproduction time that differs from a photography time of the movingimage, the generating unit

-   -   regenerates metadata such that, during a period in which the        photography frame rate is a predetermined frame rate, the period        corresponding to the characteristic amount changes at a timing        where a scene of the moving image changes and at a timing where        the photography frame rate is changed, and    -   during a period in which the photography frame rate is not the        predetermined frame rate, the period corresponding to the        characteristic amount changes at a timing where the photography        frame rate is changed.

It should be noted that each functional block according to the first toseventh embodiments may or may not be individual hardware. Functions oftwo or more functional blocks may be realized by common hardware. Eachof a plurality of functions of a single functional block may be realizedby individual hardware. Two or more functions of a single functionalblock may be realized by common hardware. In addition, each functionalblock may or may not be realized by hardware. For example, an apparatusmay include a processor and a memory storing a control program.Furthermore, functions of at least a part of functional blocks includedin the apparatus may be realized by having the processor read thecontrol program from the memory and execute the control program.

It is to be understood that the first to seventh embodiments (as wellthe modifications described above) are merely examples and thatconfigurations obtained by appropriately modifying or altering theconfigurations of the first to seventh embodiments without departingfrom the spirit and scope of the present invention are also included inthe present invention. Configurations obtained by appropriatelycombining the configurations of the first to seventh embodiments arealso included in the present invention.

According to the present disclosure, dynamic metadata and the like (acharacteristic amount of each scene) can be generated without involvingpost-production.

Other Embodiments

Embodiment(s) of the present invention can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2018-209754, filed on Nov. 7, 2018, Japanese Patent Application No.2018-239362, filed on Dec. 21, 2018, and Japanese Patent Application No.2019-033958, filed on Feb. 27, 2019, which are hereby incorporated byreference herein in their entirety.

What is claimed is:
 1. An imaging apparatus comprising: an image sensorconfigured to capture a moving image, and at least one memory and atleast one processor which function as: a determining unit configured todetermine a plurality of scenes of the moving image on a basis of a timevariation of a parameter corresponding to a frame of the moving image;an acquiring unit configured to acquire a characteristic amount of eachof the plurality of scenes; and a generating unit configured to generateinformation in which a characteristic amount acquired by the acquiringunit is associated with each of the plurality of scenes, wherein thereare a plurality of parameters corresponding to the frame, and thedetermining unit determines the plurality of scenes using a parameter inaccordance with a set photography mode among the plurality ofparameters.
 2. The imaging apparatus according to claim 1, wherein thecharacteristic amount includes a maximum brightness value of movingimage data of a scene corresponding to the characteristic amount.
 3. Theimaging apparatus according to claim 1, wherein the parameter includes amaximum brightness value of image data of the frame.
 4. The imagingapparatus according to claim 1, wherein the determining unit determinesthe plurality of scenes so that a scene change is not performed at atime position where the parameter does not vary, and a scene change isperformed at a time position where the parameter varies.
 5. The imagingapparatus according to claim 4, wherein the determining unit determinesthe plurality of scenes so that a scene change is not performed at atime position where the parameter varies by a variation amount that issmaller than a threshold, and a scene change is performed at a timeposition where the parameter varies by a variation amount that is largerthan the threshold.
 6. The imaging apparatus according to claim 1,wherein the parameter includes an imaging parameter when capturing animage of the frame.
 7. The imaging apparatus according to claim 6,wherein the imaging parameter includes at least one of ISO sensitivity,a shutter speed, an aperture value, a focus position, a focal length,white balance, and an exposure value.
 8. The imaging apparatus accordingto claim 6, wherein the imaging parameter includes at least one of anaperture value, a focus position, and a focal length.
 9. The imagingapparatus according to claim 1, wherein in a case where an image of aframe includes an in-focus region and an out-of-focus region, thedetermining unit determines the plurality of scenes on a basis of theparameter corresponding to only the in-focus region among the in-focusregion and the out-of-focus region.
 10. The imaging apparatus accordingto claim 1, further comprising an output interface for connecting to anexternal apparatus, wherein the at least one memory and the at least oneprocessor further function as: an output control unit configured toperform control so as to output data of the moving image and theinformation in association with each other, and the output control unitperforms control so as to output the data of the moving image in astream format from the output interface to the external apparatus. 11.The imaging apparatus according to claim 1, wherein the acquiring unitacquires a maximum brightness value of image data corresponding to anin-focus region of an image of a frame with respect to each of aplurality of frames of the moving image, and includes a maximum value oftwo or more maximum brightness values acquired with respect to a sceneof the moving image in the characteristic amount of the scene.
 12. Theimaging apparatus according to claim 1, wherein the determining unitdetermines the plurality of scenes so that a scene change is notperformed independent of a time variation of the parameter at a timeposition where an in-focus region of an image of the frame narrows. 13.The imaging apparatus according to claim 1, wherein the determining unitdoes not determine the plurality of scenes in a case where a specificphotography mode is set.
 14. The imaging apparatus according to claim 1,wherein the parameter includes an average value of a maximum brightnessvalue of image data of a corresponding frame that is the framecorresponding to the parameter and a maximum brightness value of one ormore frames that are time-sequential with respect to the correspondingframe.
 15. The imaging apparatus according to claim 1, wherein thecharacteristic amount is dynamic metadata specified in SMPTE ST 2094.16. A control method of an imaging apparatus, the control methodcomprising: a capturing step of capturing a moving image; a determiningstep of determining a plurality of scenes of the moving image on a basisof a time variation of a parameter corresponding to a frame of themoving image; an acquiring step of acquiring a characteristic amount ofeach of the plurality of scenes; and a generating step of generatinginformation in which a characteristic amount acquired in the acquiringstep is associated with each of the plurality of scenes, wherein thereare a plurality of parameters corresponding to the frame, and in thedetermining step, the plurality of scenes is determined using aparameter in accordance with a set photography mode among the pluralityof parameters.
 17. A non-transitory computer readable medium that storesa program, wherein the program causes a computer to execute a controlmethod of an imaging apparatus, the control method comprising: acapturing step of capturing a moving image; a determining step ofdetermining a plurality of scenes of the moving image on a basis of atime variation of a parameter corresponding to a frame of the movingimage; an acquiring step of acquiring a characteristic amount of each ofthe plurality of scenes; and a generating step of generating informationin which a characteristic amount acquired in the acquiring step isassociated with each of the plurality of scenes, wherein there are aplurality of parameters corresponding to the frame, and in thedetermining step, the plurality of scenes is determined using aparameter in accordance with a set photography mode among the pluralityof parameters.